The AdaCore Blog http://blog.adacore.com/ An insight into the AdaCore ecosystem en-us Mon, 20 Nov 2017 19:10:16 +0000 Mon, 20 Nov 2017 19:10:16 +0000 Physical Units Pass the Generic Test http://blog.adacore.com/physical-units-pass-the-generic-test Thu, 16 Nov 2017 07:30:00 +0000 Yannick Moy http://blog.adacore.com/physical-units-pass-the-generic-test

The support for physical units in programming languages is a long-standing issue, which very few languages have even attempted to solve. This issue was mostly solved for Ada in 2012 by our colleagues Ed Schonberg and Vincent Pucci, who introduced special aspects for specifying physical dimensions on types. An aspect Dimension_System allows the programmer to define a new system of physical units, while an aspect Dimension allows setting the dimensions of a given subtype in the dimension system of its parent type. The dimension system in GNAT is completely checked at compile time, with no impact on the executable size or execution time, and it offers a number of facilities for defining units, managing fractional dimensions and printing out dimensioned quantities. For details, see the article "Implementation of a simple dimensionality checking system in Ada 2012" presented at the ACM SIGAda conference HILT 2012 (also attached below).

The GNAT dimension system did not attempt to deal with generics though. As noted in the previous work by Grein, Kazakov and Wilson in "A survey of Physical Units Handling Techniques in Ada":

The conflict between the requirements 1 [Compile-Time Checks] and 2 [No Memory / Speed Overhead at Run-Time] on one side and requirement 4 [Generic Programming] on the other is the source of the problem of dimension handling.

So the solution in GNAT solved 1 and 2 while allowing generic programming but leaving it unchecked for dimensionality correctness. Here is the definition of generic programming by Grein, Kazakov and Wilson:

Here generic is used in a wider sense, as an ability to write code dealing with items of types from some type sets.  In our case it means items of different dimension. For instance, it should be possible to write a dimension-aware integration program, which would work for all valid combinations of dimensions.

This specific issue of programming a generic integration over time that would work for a variety of dimensions was investigated earlier this year by our partners from Technical University of Munich for their Glider autopilot software in SPARK. Working with them, we found a way to upgrade the dimensionality analysis in GNAT to support generic programming, which recently has been implemented in GNAT.

The goal was to apply dimensionality analysis on the instances of a generic, and to preserve dimension as much as possible across type conversions. In our upgraded dimensionality analysis in GNAT, a conversion from a dimensioned type (say, a length) to its dimensionless base type (the root of the dimension system) now preserves the dimension (length here), but will look like valid Ada code to any other Ada compiler. Which makes it possible to define a generic function Integral as follows:

    generic
        type Integrand_Type is digits <>;
        type Integration_Type is digits <>;
        type Integrated_Type is digits <>;
    function Integral (X : Integrand_Type; T : Integration_Type) return Integrated_Type;

    function Integral (X : Integrand_Type; T : Integration_Type) return Integrated_Type is
    begin
       return Integrated_Type (Mks_Type(X) * Mks_Type(T));
    end Integral;

We are using above the standard Mks_Type defined in System.Dim.Mks as root type, but we could do the same with the root of a user-defined dimension system. The generic code is valid Ada since the arguments X and T are converted into the same type (here Mks_Type), in order to multiply them without compilation error. The result, which is now of type Mks_Type, is then converted to the final type Integrated_Type. With the original dimensionality analysis in GNAT, dimensions were lost during these type conversions. 

However, with the upgraded analysis in GNAT, a conversion to the root type indicates that the dimensions of X and T have to be preserved and tracked when converting both to and from the root type Mks_Type. With this, still using the standard types defined in System.Dim.Mks, we can define an instance of this generic that integrates speed over time:

    function Velocity_Integral is new Integral(Speed, Time, Length);

The GNAT-specific dimensionality analysis will perform additional checks for correct dimensionality in all such generic instances, while for any other Ada compiler this program still passes as valid (but not dimensionality-checked) Ada code. For example, for an invalid instance as:

    function Bad_Velocity_Integral is new Integral(Speed, Time, Mass);

GNAT issues the error:

dims.adb:10:05: instantiation error at line 5
dims.adb:10:05: dimensions mismatch in conversion
dims.adb:10:05: expression has dimension [L]
dims.adb:10:05: target type has dimension [M]

One subtlety that we faced when developing the Glider software at Technical University of Munich was that, sometimes, we do want to convert a value from a dimensioned type into another dimensioned type. This was the case in particular because we defined our own dimension system in which angles had their own dimension to verify angular calculations, which worked well most of the time.

However, the angle dimension must be removed when multiplying an angle with a length, which produces an (arc) length, or when using an existing trigonometric function that expects a dimensionless argument. In theses cases, a simple type conversion to the dimensionless root type is not enough, because now the dimension of the input is preserved. We found two solutions to this problem:

  • either define the root of our dimension system as a derived type from a parent type, say Base_Unit_Type, and convert to/from Base_Unit_Type to remove dimensions; or
  • explicitly insert conversion coefficients into the equations with dimensions such that the dimensions do cancel out as required.

For example, our use of an explicit Angle_Type with its own dimension (denoted A) first seemed to cause trouble because of conversions such as this one:

Distance := 2.0 * EARTH_RADIUS * darc; -- expected L, found L.A

where darc is of Angle_Type (dimension A) and EARTH_RADIUS of Length_Type (dimension L). First, we escaped the unit system as follows:

Distance := 2.0 * EARTH_RADIUS * Unit_Type(Base_Unit_Type(darc));

However, this bypasses the dimensionality checking system and can lead to dangerous mixing of physical dimensions. It would be possible to accidentally turn a temperature into a distance, without any warning. A safer way to handle this issue is to insert the missing units explicitly:

Distance := 2.0 * EARTH_RADIUS * darc * 1.0/Radian;

Here, Radian is the unit of Angle_Type, which we need to get rid of to turn an angle into a distance. In other words, the last term represents a coefficient with the required units to turn an angle into a distance. Thus, darc*1.0/Radian still carries the same value as darc, but is dimensionless as required per the equation, and GNAT can perform a dimensionality analysis also in such seemingly dimensionality-defying situations.

Moreover, this solution is less verbose than converting to the base unit type and then back. In fact, it can be made even shorter:

Distance := 2.0 * EARTH_RADIUS * darc/Radian;

With its improved dimensionality analysis, GNAT Pro 18 has solved the conflict between requirements 1 [Compile-Time Checks] and 2 [No Memory / Speed Overhead at Run-Time] on one side and requirement 4 [Generic Programming] on the other side, hopefully making Grein, Kazakov and Wilson happier! The dimensionality analysis in GNAT is a valuable feature for programs that deal with physical units. It increases readability by making dimensions more explicit and it reduces programming errors by checking the dimensions for consistency. For example, we used it on the StratoX Weather Glider from Technical University of Munich, as well as the RESSAC User Case, an example of autonomous vehicle development used as challenge for certification.

For more information on the dimensionality analysis in GNAT, see the GNAT User's Guide. In particular, the new rules that deal with conversions are at the end of the section, and we copy them verbatim below:

  The dimension vector of a type conversion T(expr) is defined as follows, based on the nature of T:
 -  If T is a dimensioned subtype then DV(T(expr)) is DV(T) provided that either expr is dimensionless or DV(T) = DV(expr). The conversion is illegal if expr is dimensioned and DV(expr) /= DV(T). Note that vector equality does not require that the corresponding Unit_Names be the same.
    As a consequence of the above rule, it is possible to convert between different dimension systems that follow the same international system of units, with the seven physical components given in the standard order (length, mass, time, etc.). Thus a length in meters can be converted to a length in inches (with a suitable conversion factor) but cannot be converted, for example, to a mass in pounds.
 -  If T is the base type for expr (and the dimensionless root type of the dimension system), then DV(T(expr)) is DV(expr). Thus, if expr is of a dimensioned subtype of T, the conversion may be regarded as a “view conversion” that preserves dimensionality.
    This rule makes it possible to write generic code that can be instantiated with compatible dimensioned subtypes. The generic unit will contain conversions that will consequently be present in instantiations, but conversions to the base type will preserve dimensionality and make it possible to write generic code that is correct with respect to dimensionality.
 -  Otherwise (i.e., T is neither a dimensioned subtype nor a dimensionable base type), DV(T(expr)) is the empty vector. Thus a dimensioned value can be explicitly converted to a non-dimensioned subtype, which of course then escapes dimensionality analysis.

Thanks to Ed Schonberg and Ben Brosgol from AdaCore for their work on the design and implementation of this enhanced dimensionality analysis in GNAT.

]]>
Make with Ada 2017: Brushless DC Motor Controller http://blog.adacore.com/make-with-ada-2017-brushless-dc-motor-controller Tue, 14 Nov 2017 14:33:39 +0000 Jonas Attertun http://blog.adacore.com/make-with-ada-2017-brushless-dc-motor-controller

Not long after my first experience with the Ada programming language I got to know about the Make With Ada 2017 contest. And, just as it seemed, it turned out to be a great way to get a little bit deeper into the language I had just started to learn

The ada-motorcontrol project involves the design of a BLDC motor controller software platform written in Ada. These types of applications often need to be run fast and the core control software is often tightly connected to the microcontroller peripherals. Coming from an embedded systems perspective with C as the reference language, the initial concerns were if an implementation in Ada actually could meet these requirements.

It turned out, on the contrary, that Ada is very capable considering both these requirements. In particular, accessing peripherals on the STM32 with help of the Ada_Drivers_Library really made using the hardware related operations even easier than using the HAL written in C by ST.

Throughout the project I found uses for many of Ada’s features. For example, the representation clause feature made it simple to extract data from received (and to put together the transmit) serial byte streams. Moreover, contract based programming and object oriented concepts such as abstracts and generics provided means to design clean and easy to use interfaces, and a well organized project.

One of the objectives of the project was to provide a software platform to help developing various motor control applications, with the core functionality not being dependent on some particular hardware. Currently however it only supports a custom inverter board, since unfortunately I found that the HAL provided in Ada_Drivers_Library was not comprehensive enough to support all the peripheral features used. But the software is organized such as to keep driver dependent code separated. To put this to test, I welcome contributions to add support for other inverter boards. A good start would be the popular VESC-board.


Ada Motor Controller Project Log:

Motivation

The recent advances in electric drives technologies (batteries, motors and power electronics) has led to increasingly higher output power per cost, and power density. This in turn has increased the performance of existing motor control applications, but also enabled some new - many of them are popular projects amongst diyers and makers, e.g. electric bike, electric skateboard, hoverboard, segway etc. 

On a hobby-level, the safety aspects related to these is mostly ignored. Professional development of similar applications, however, normally need to fulfill some domain specific standards putting requirements on for example the development process, coding conventions and verification methods. For example, the motor controller of an electric vehicle would need to be developed in accordance to ISO 26262, and if the C language is used, MISRA-C, which defines a set of programming guidelines that aims to prevent unsafe usage of the C language features. 

Since the Ada programming language has been designed specifically for safety critical applications, it could be a good alternative to C for implementing safe motor controllers used in e.g. electric vehicle applications. For a comparison of MISRA-C and Ada/SPARK, see this report. Although Ada is an alternative for achieving functional safety, during prototyping it is not uncommon that a mistake leads to destroyed hardware (burned motor or power electronics). Been there, done that! The stricter compiler of Ada could prevent such accidents. 

Moreover, while Ada is not a particularly "new" language, it includes more features that would be expected by a modern language, than is provided by C. For example, types defined with a specified range, allowing value range checks already during compile time, and built-in multitasking features. Ada also supports modularization very well, allowing e.g. easy integration of new control interfaces - which is probably the most likely change needed when using the controller for a new application. 

This project should consist of and fulfill:

  • Core software for controlling brushless DC motors, mainly aimed at hobbyists and makers.
  • Support both sensored and sensorless operation.
  • Open source software (and hardware).
  • Implementation in Ada on top of the Ravenscar runtime for the stm32f4xx.
  • Should not be too difficult to port to another microcontroller.

And specifically, for those wanting to learn the details of motor control, or extend with extra features:

  • Provide a basic, clean and readable implementation.
  • Short but helpful documentation. 
  • Meaningful and flexible logging.
  • Easy to add new control interfaces (e.g. CAN, ADC, Bluetooth, whatever).

Hardware

The board that will be used for this project is a custom board that I previously designed with the intent to get some hands-on knowledge in motor control. It is completely open source and all project files can be found on GitHub

  • Microcontroller STM32F446, ARM Cortex-M4, 180 MHz, FPU
  • Power MOSFETs 60 V
  • Inline phase current sensing
  • PWM/PPM control input
  • Position sensor input as either hall or quadrature encoder
  • Motor and board temp sensor (NTC)
  • Expansion header for UART/ADC/DAC/SPI/I2C/CAN

It can handle power ranges in the order of what is required by an electric skateboard or electric bike, depending on the used battery voltage and cooling. 

There are other inverter boards with similar specifications. One very popular is the VESC by Benjamin Vedder. It is probably not that difficult to port this project to work on that board as well.

Rough project plan

I thought it would be appropriate to write down a few bullets of what needs to be done. The list will probably grow...

  • Create a port of the Ravenscar runtime to the stm32f446 target on the custom board
  • Add stm32f446 as a device in the Ada Drivers Library
  • Get some sort of hello world application running to show that stuff works
  • Investigate and experiment with interrupt handling with regards to overhead
  • Create initialization code for all used mcu peripherals
  • Sketch the overall software architecture and define interfaces
  • Implementation
  • Documentation...

Support for the STM32F446

The microprocessor that will be used for this project is the STM32F446. In the current version of the Ada Drivers Library and the available Ravenscar embedded runtimes, there is no explicit support for this device. Fortunately, it is very similar to other processors in the stm32f4-family, so adding support for stm32f446 was not very difficult once I understood the structure of the repositories. I forked these and added them as submodules in this project's repo

Compared to the Ravenscar runtimes used by the discovery-boards, there are differences in external oscillator frequency, available interrupt vectors and memory sizes. Otherwise they are basically the same. 

An important tool needed to create the new driver and runtime variants is svd2ada. It generates device specification and body files in ada based on an svd file (basically xml) that describes what peripherals exist, how their registers look like, their address', existing interrupts, and stuff like that. It was easy to use, but a little bit confusing how flags/switches should be set when generating driver and runtime files. After some trail and error I think I got it right. I created a Makefile for generating all these file with correct switches.

I could not find an svd-file for the stm32f446 directly from ST, but found one on the internet. It was not perfect though. Some of the source code that uses the generated data types seems to make assumptions on the structure of these types. Depending on how the svd file looks, svd2ada may or may not generate them in the expected way. There were also other missing and incorrect data in the svd file, so I had to manually correct these. There are probably additional issues that I have not found yet...

It is alive!

I made a very simple application consisting of a task that is periodically delayed and toggles the two leds on the board each time the task resumes. The leds toggles with the expected period, so the oscillator seems to be initialized correctly. 

Next up I need to map the different mcu pins to the corresponding hardware functionality and try to initialize the needed peripherals correctly. 

The control algorithm and its use of peripherals

There are several methods of controlling brushless motors, each with a specific use case. As a first approach I will implement sensored FOC, where the user requests a current value (or torque value). 

To simplify, this method can be divided into the following steps, repeated each PWM period (typically around 20 kHz):

  1. Sample the phase currents
  2. Transform the values into a rotor fixed reference frame
  3. Based on the requested current, calculate a new set of phase voltages
  4. Transform back to the stator's reference frame
  5. Calculate PWM duty cycles as to create the calculated phase voltages

Fortunately, the peripherals of the stm32f446 has a lot of features that makes this easier to implement. For example, it is possible to trigger the ADC directly from the timers that drives the PWM. This way the sampling will automatically be synchronized with the PWM cycle. Step 1 above can thus be started immediately as the ADC triggers the corresponding conversion-complete-interrupt. In fact, many existing implementations perform all the steps 1-to-6 completely within an ISR. The reason for this is simply to reduce any unnecessary overhead since the performed calculations is somewhat lengthy. The requested current is passed to the ISR via global variables. 

I would like to do this the traditional way, i.e. to spend as little time as possible in the ISR and trigger a separate Task to perform all calculations. The sampled current values and the requested current shall be passed via Protected Objects. All this will of course create more overhead. Maybe too much? Need to be investigated.

PWM and ADC is up and running

I have spent some time configuring the PWM and ADC peripherals using the Ada Drivers Library. All in all it went well, but I had to do some smaller changes to the drivers to make it possible to configure the way I wanted. 

  • PWM is complementary output, center aligned with frequency of 20 kHz
  • PWM channels 1 to 3 generates the phase voltages
  • PWM channel 4 is used to trigger the ADC, this way it is possible to set where over the PWM period the sampling should occur
  • By default the sampling occurs in the middle of the positive waveform (V7)
  • The three ADC's are configured to Triple Multi Mode, meaning they are synchronized such that each sampled phase quantity is sampled at the same time. 
  • Phase currents and voltages a,b,c are mapped to the injected conversions, triggered by the PWM channel 4
  • Board temperature and bus voltage is mapped to the regular conversions triggered by a timer at 14 kHz
  • Regular conversions are moved to a volatile array using DMA automatically after the conversions complete
  • ADC create an interrupt after the injected conversions are complete

The drivers always assumed that the PWM outputs are mapped to a certain GPIO, so in order to configure the trigger channel I had to add a new procedure to the drivers. Also, the Scan Mode of the ADCs where not set up correctly for my configuration, and the config of injected sequence order was simply incorrect. I will send a pull request to get these changes merged with the master branch. 

Interrupt overhead/latency

As was described in previous posts the structure used for the interrupt handling is to spend minimum time in the interrupt context and to signal an awaiting task to perform the calculations, which executes at a software priority level with interrupts fully enabled. The alternative method is to place all code in the interrupt context. 

This Ada Gem and its following part describes two different approaches for doing this type of task synchronization. Both use a protected procedure as the interrupt handler but signals the awaiting task in different ways. The first uses an entry barrier and the second a Suspension Object. The idiom using the entry barrier has the advantage that it can pass data as an integrated part of the signaling, while the Suspension Object behaves more like a binary semaphore. 

For the ADC conversion complete interrupt, I tested both methods. The protected procedure used as the ISR read the converted values consisting of six uint16. For the entry barrier method these where passed to the task using an out-parameter. When using the second method the task needed to collect the sample data using a separate function in the protected object.

Overhead in this context I define as the time from that the ADC generates the interrupt, to the time the event triggered task starts running. This includes, first, an isr-wrapper that is a part of the runtime which then calls the installed protected procedure, and second, the execution time of the protected procedure which reads the sampled data, and finally, the signaling to the awaiting task. 

I measured an approximation of the overhead by setting a pin high directly in the beginning of the protected procedure and then low by the waiting task directly when waking up after the signaling. For the Suspension Object case the pin was set low after the read data function call, i.e. for both cases when the sampled data was copied to the task. The code was compiled with the -O3 flag. 

The first idiom resulted in an overhead of ~8.4 us, and the second ~10 us. This should be compared to the period of the PWM which at 20 kHz is 50 us. Obviously the overhead is not negligible, so I might consider using the more common approach for motor control applications of having the current control algorithm in the interrupt context instead. However, until the execution time of the algorithm is known, the entry barrier method will be assumed... 

Note: "Overhead" might be the wrong term since I don't know if during the time measured the cpu was really busy. Otherwise it should be called latency I think...

Purple: Center aligned PWM at 50 % duty where the ADC triggers in the center of the positive waveform. Yellow: Pin state as described above. High means time of overhead/latency.

Reference frames

A key benefit of the FOC algorithm is that the actual control is performed in a reference frame that is fixed to the rotor. This way the sinusoidal three phase currents, as seen in the stator's reference frame, will instead be represented as two DC values, assuming steady state operation. The transforms used (Clarke and Park) requires that the angle between the rotor and stator is known. As a first step I am using a quadrature encoder since that provides a very precise measurement and very low overhead due to the hardware support of the stm32. 

Three types has been defined, each representing a particular reference frame: Abc, Alfa_Beta and Dq. Using the transforms above one can simply write:

declare
   Iabc  : Abc;  --  Measured current (stator ref)
   Idq   : Dq;   --  Measured current (rotor ref)
   Vdq   : Dq;   --  Calculated output voltage (rotor ref)
   Vabc  : Abc;  --  Calculated output voltage (stator ref)
   Angle : constant Float := ...;
begin
   Idq := Iabc.Clarke.Park(Angle);

   --  Do the control...

   Vabc := Vdq.Park_Inv(Angle).Clarke_Inv;
end;

Note that Park and Park_Inv both use the same angle. To be precise, they
both use Sin(Angle) and Cos(Angle). Now, at first, I simply implemented
these by letting each transform calculate Sin and Cos locally. Of
course, that is a waste for this particular application. Instead, I
defined an angle object that when created also computed Sin and Cos of
the angle, and added versions of the transforms to use these
"ahead-computed" values instead.

declare
   --  Same...

   Angle : constant Angle_Obj := Compose (Angle_Rad); 
   --  Calculates Sin and Cos
begin
   Idq := Iabc.Clarke.Park(Angle);

   --  Do the control...

   Vabc := Vdq.Park_Inv(Angle).Clarke_Inv;
end;

This reduced the execution time somewhat (not as much as I thought, though), since the trigonometric functions are the heavy part. Using lookup table based versions instead of the ones provided by Ada.Numerics might be even faster...

It spins!

The main structure of the current controller is now in place. When a button on the board is pressed the sensor is aligned to the rotor by forcing the rotor to a known angle. Currently, the requested q-current is set by a potentiometer. 

As of now, it is definitely not tuned properly, but it at least it shows that the general algorithm is working as intended. 

In order to make this project easier to develop on, both for myself and any other users, I need to add some logging and tuning capabilities. This should allow a user to change and/or log variables in the application (e.g. control parameters) while the controller is running. I have written a tool for doing this (over serial) before, but then in C. It would be interesting to rewrite it in Ada. 

Contract Based Programming

So far, I have not used this feature much. But when writing code for the logging functionality I ran into a good fit for it. 

I am using Consistent Overhead Byte Stuffing (COBS) to encode the data sent over uart. This encoding results in unambiguous packet framing regardless of packet content, thus making it easy for receiving applications to recover from malformed packets. The packets are separated by a delimiter (value 0 in this case), making it easy to synchronize the receiving parser. The encoding ensures that the encoded packet itself does not contain the delimiter value. 

A good feature of COBS is that given that the raw data length is less than 254, then the overhead due to the encoding is always exactly one byte. I could of course simply write this fact as a comment to the encode/decode functions, allowing the user to make this assumption in order to simplify their code. A better way could be to write this condition as contracts. 

   Data_Length_Max : constant Buffer_Index := 253;

   function COBS_Encode (Input : access Data)
                         return Data
   with
      Pre => Input'Length <= Data_Length_Max,
      Post => (if Input'Length > 0 then
                  COBS_Encode'Result'Length = Input'Length + 1
               else
                  Input'Length = COBS_Encode'Result'Length);

   function COBS_Decode (Encoded_Data : access Data)
                         return Data
   with
      Pre => Encoded_Data'Length <= Data_Length_Max + 1,
      Post => (if Encoded_Data'Length > 0 then
                  COBS_Decode'Result'Length = Encoded_Data'Length - 1
               else
                  Encoded_Data'Length = COBS_Decode'Result'Length);

Logging and Tuning

I just got the logging and tuning feature working. It is an Ada-implementation using the protocol as used by a previous project of mine, Calmeas. It enables the user to log and change the value of variables in the application, in real-time. This is very helpful when developing systems where the debugger does not have a feature of reading and writing to memory while the target is running. 

The data is sent and received over uart, encoded by COBS. The interfaces of the uart and cobs packages implements an abstract stream type, meaning it is very simple to change the uart to some other media, and that e.g. cobs can be skipped if wanted. 

Example

The user can simply do the following in order to get the variable V_Bus_Log loggable and/or tunable:

V_Bus_Log  : aliased Voltage_V;
...
Calmeas.Add (Symbol      => V_Bus_Log'Access,
             Name        => "V_Bus",
             Description => "Bus Voltage [V]");

It works for (un)signed integers of size 8, 16 and 32 bits, and for floats. 

After adding a few variables, and connecting the target to the gui:

As an example, this could be used to tune the current controller gains:

As expected, the actual current comes closer to the reference as the gain increases

As of now, the tuning is not done in a "safe" way. The writing to added symbols is done by the separate task named Logger, simply by doing unchecked writes to the address of the added symbol, one byte at a time. At the same time the application is reading the symbol's value from another task with higher prio. The optimal way would be to pass the value through a protected type, but since the tuning is mostly for debugging purposes, I will make it the proper way later on...

Note that the host GUI is not written in Ada (but Python), and is not itself a part of this project. 

Architecture overview

Here is a figure showing an overview of the software:

Summary

This project involves the design of a software platform that provides a good basis when developing motor controllers for brushless motors. It consist of a basic but clean and readable implementation of a sensored field oriented control algorithm. Included is a logging feature that will simplify development and allows users to visualize what is happening. The project shows that Ada successfully can be used for a bare-metal project that requires fast execution. 

The design is, thanks to Ada's many nice features, much easier to understand compared to a lot of the other C-implementations out there, where, as a worst case, everything is done in a single ISR. The combination of increased design readability and the strictness of Ada makes the resulting software safer and simplifies further collaborative development and reuse. 

Some highlights of what has been done:

  • Porting of the Ravenscar profiles to a custom board using the STM32F446
  • Adding support for the STM32F446 to Ada_Drivers_Library project
  • Adding some functionality to Ada_Drivers_Library in order to fully use all peripheral features
  • Fixing a bug in Ada_Drivers_Library related to a bit more advanced ADC usage
  • Written HAL-isch packages so that it is easy to port to another device than STM32
  • Written a communication package and defined interfaces in order to make it easier to add control inputs.
  • Written a logging package that allows the developer to debug, log and tune the application in real-time.
  • Implemented a basic controller using sensored field oriented control
  • Well documented specifications with a generated html version

Future plans:

  • Add hall sensor support and 6-step block commutation
  • Add sensorless operation
  • Add CAN support (the pcb has currently no transceiver, though)
  • SPARK proving
  • Write some additional examples showing how to use the interfaces.
  • Port the software to the popular VESC-board.

]]>
Highlighting Ada with Libadalang http://blog.adacore.com/highlighting-ada-with-libadalang Tue, 08 Aug 2017 12:30:00 +0000 Pierre-Marie de Rodat http://blog.adacore.com/highlighting-ada-with-libadalang

While we are working very hard on semantic analysis in Libadalang, it is already possible to leverage its lexical and syntactic analyzers. A useful example for this is a syntax highlighter.

In the context of programming languages, syntax highlighters make it easier for us, humans, to read source code. For example, formatting keywords with a bold font and a special color, while leaving identifiers with a default style, enables a quick understanding of a program’s structure. Syntax highlighters are so useful that they’re integrated into daily developer tools:

  • most “serious” code editors provide highlighters for tons of programming languages (C, Python, Ada, OCaml, shell scripts), markup languages (XML, HTML, BBcode), DSLs (Domain Specific Languages: SQL, TeX), etc.
  • probably all online code browsers ship syntax highlighters at least for mainstream languages: GitHub, Bitbucket, GitLab, …

From programming language theory, that many of us learned in engineering school (some even with passion!), we can distinguish two highlighting levels:

  1. Tokens(lexeme)-based highlighting. Each token is given a style from its token kind (keyword, comment, integer literal, identifier, …).

  2. Syntax-based highlighting. This one is higher-level: for example, give a special color for identifiers that give their name to functions and another color for identifiers that give their name to type declarations.

Most syntax highlighting engines, for instance pygments’s or vim’s, are based on regular expressions, which don’t offer to highlighter writers the same formalism as what we just described. Generally, regular expressions enable us to create an approximation of the two highlighting levels, which yields something a bit fuzzy. On the other hand, it is much easier to write such highlighters and to get pretty results quickly, compared to a fully fledged lexer/parser, which is probably why regular expressions are so popular in this context.

But here we already have a full lexer/parser for the Ada language: Libadalang. So why not use it to build a “perfect” syntax highlighter? This is going to be the exercise of the day. All blog posts so far only demonstrated the use of Libadalang’s Python API; Libadalang is primarily an Ada library, so let’s use its Ada API for once!

One disclaimer, first: the aim of this post is not to say that the world of syntax highlighters is broken and should be re-written with compiler-level lexers and parsers. Regexp-based highlighters are a totally fine compromise in contexts such as text editors; here we just demonstrate how Libadalang can be used to achieve a similar goal, but keep in mind that the result is not technically equivalent. For instance, what we will do below will require valid input Ada sources and will only work one file at a time, unlike editors that might need to work on smaller granularity items to keep the UI responsive, which is more important in this context than correctness.

Okay so, how do we start?

The first thing to do as soon as one wants to use Libadalang is to create an analysis context: this is an object that will enclose the set of sources files to process.

with Libadalang.Analysis;

package LAL renames Libadalang.Analysis;
Ctx : LAL.Analysis_Context := LAL.Create;

Good. At the end of our program, we need to release the resources that were allocated in this context:

LAL.Destroy (Ctx);

Now everything interesting must happen in between. Let’s ask Libadalang to parse a source file:

Unit : LAL.Analysis_Unit := LAL.Get_From_File
   (Ctx, "my_source_file.adb", With_Trivia => True);

The analysis of a source file yields what we call in Libadalang an analysis unit. This unit is tied to the context used to create it.

Here, we also enable a peculiar analysis option: With_Trivia tells Libadalang not to discard “trivia” tokens. Inspired by the Roslyn Compiler, what we call a trivia token is a token that is ignored when parsing source code. In Ada as in most (all?) programming languages, comments are trivia: developers are allowed to put any number of them anywhere between two tokens, this will not change the validity of the program nor its semantics. Because of this most compiler implementations just discard them: keeping them around would hurt performance for no gain. Libadalang is designed for all kinds of tools, not only compilers, so we give the choice to the user of whether or not to keep trivia around.

What we are trying to do here is to highlight a source file. We want to highlight comments as well, so we need to ask to preserve trivia.

At this point, a real world program would have to check if parsing happened with no error. We are just playing here, so we’ll skip that, but you can have a look at the Libadalang.Analysis.Has_Diagnostic and Libadalang.Analysis.Diagnostics functions if you want to take care of this.

Fine, so we assume parsing went well and now we just have to go through tokens and assign a specific style to each of them. First, let’s have a look at the various token-related data types in Libadalang we have to deal with:

  • LAL.Token_Type: reference to a token/trivia in a specific analysis unit. Think of it as a cursor in a standard container. There is one special value: No_Token which, as you may guess, is used to represent the end of the token stream or just an invalid reference, like a null access.

  • LAL.Token_Data_Type: holder for the data related to a specific token. Namely: token kind, whether it’s trivia, index in the token/trivia stream and source location range.

  • Libadalang.Lexer.Token_Data_Handlers.Token_Index: a type derived from Integer to represent the indexes in token/trivia streams.

Then let’s define holders to annotate the token stream:

type Highlight_Type is (Text, Comment, Keyword, Block_Name, ...);

Instead of directly assigning colors to the various token kinds, this enumeration defines categories for highlighting. This makes it possible to provide different highlighting styles later: one set of colors for a white background, and another one for a black background, for example.

subtype Token_Index is
   Libadalang.Lexer.Token_Data_Handlers.Token_Index;

type Highlight_Array is
   array (Token_Index range <>) of Highlight_Type;

type Highlights_Holder (Token_Count, Trivia_Count : Token_Index) is
record
   Token_Highlights  : Highlight_Array (1 .. Token_Count);
   Trivia_Highlights : Highlight_Array (1 .. Trivia_Count);
end record;

In Libadalang, even though tokens and trivia make up a logical interleaved stream, they are stored as two separate streams, hence the need for two arrays. So here is a procedure to make the annotating process easier:

procedure Set
  (Highlights : in out Highlights_Holder;
   Token      : LAL.Token_Data_Type;
   HL         : Highlight_Type)
is
   Index : constant Token_Index := LAL.Index (Token);
begin
   if LAL.Is_Trivia (Token) then
      Highlights.Trivia_Highlights (Index) := HL;
   else
      Highlights.Token_Highlights (Index) := HL;
   end if;
end Set;

Now let’s start the actual highlighting! We begin with the token-based one as described earlier.

Basic_Highlights : constant
  array (Libadalang.Lexer.Token_Kind) of Highlight_Type :=
 (Ada_Identifier => Identifier,
      Ada_All .. Ada_Return
    | Ada_Elsif | Ada_Reverse
    | -- ...
      => Keyword,
    --  ...
  );

The above declaration associate a highlighting class for each token kind defined in Libadalang.Lexer. The only work left is to determine highlighting classes iterating on each token in Unit:

Token : LAL.Token_Type := LAL.First_Token (Unit);

while Token /= LAL.No_Token loop
   declare
      TD : constant LAL.Token_Data_Type := LAL.Data (Token);
      HL : constant Highlight_Type :=
         Basic_Highlights (LAL.Kind (TD));
   begin
      Set (Highlights, TD, HL);
   end;
   Token := LAL.Next (Token);
end loop;

Easy, right? Once this code has run, we already have a pretty decent highlighting for our analysis unit! The second pass is just a refinement that uses syntax as described at the top of this blog post:

function Syntax_Highlight
  (Node : access LAL.Ada_Node_Type'Class) return LAL.Visit_Status;
 
LAL.Traverse (LAL.Root (Unit), Syntax_Highlight'Access);

LAL.Traverse will traverse Unit’s syntax tree (AST) and call the Syntax_Highlight function on each node. This function is a big dispatcher on the kind of the visited node:

function Syntax_Highlight
  (Node : access LAL.Ada_Node_Type'Class) return LAL.Visit_Status
is
   procedure Highlight_Block_Name
      (Name : access LAL.Name_Type'Class) is
   begin
      Highlight_Name (Name, Block_Name, Highlights);
   end Highlight_Block_Name;
begin
   case Node.Kind is
      when LAL.Ada_Subp_Spec =>
         declare
            Subp_Spec : constant LAL.Subp_Spec :=
               LAL.Subp_Spec (Node);

            Params : constant LAL.Param_Spec_Array_Access :=
               Subp_Spec.P_Node_Params;

         begin
            Highlight_Block_Name
              (Subp_Spec.F_Subp_Name, Highlights);
            Highlight_Type_Expr
              (Subp_Spec.F_Subp_Returns, Highlights);
            for Param of Params.Items loop
               Highlight_Type_Expr
                 (Param.F_Type_Expr, Highlights);
            end loop;
         end;

      when LAL.Ada_Subp_Body =>
         Highlight_Block_Name
           (LAL.Subp_Body (Node).F_End_Id, Highlights);

      when LAL.Ada_Type_Decl =>
         Set (Highlights,
              LAL.Data (Node.Token_Start),
              Keyword_Type);
         Highlight_Block_Name
           (LAL.Type_Decl (Node).F_Type_Id, Highlights);

      when LAL.Ada_Subtype_Decl =>
         Highlight_Block_Name
           (LAL.Subtype_Decl (Node).F_Type_Id, Highlights);

      --  ...
   end case;
   return LAL.Into;
end Syntax_Highlight;

Depending on the nature of the AST node to process, we apply specific syntax highlighting rules. For example, the first one above: for subprogram specifications (Subp_Spec), we highlight the name of the subprogram as a “block name” while we highlight type expressions for the return type and the type of all parameters as “type expressions”. Let’s go deeper: how do we highlight names?

procedure Highlight_Name
  (Name       : access LAL.Name_Type'Class;
   HL         : Highlight_Type;
   Highlights : in out Highlights_Holder) is
begin
   if Name = null then
      return;
   end if;

   case Name.Kind is
      when LAL.Ada_Identifier | LAL.Ada_String_Literal =>
         --  Highlight the only token that this node has
         declare
            Tok : constant LAL.Token_Type :=
              LAL.Single_Tok_Node (Name).F_Tok;
         begin
            Set (Highlights, LAL.Data (Tok), HL);
         end;

      when LAL.Ada_Dotted_Name =>
         --  Highlight both the prefix, the suffix and the
         --  dot token.

         declare
            Dotted_Name : constant LAL.Dotted_Name :=
               LAL.Dotted_Name (Name);
            Dot_Token   : constant LAL.Token_Type :=
               LAL.Next (Dotted_Name.F_Prefix.Token_End);
         begin
            Highlight_Name
              (Dotted_Name.F_Prefix, HL, Highlights);
            Set (Highlights, LAL.Data (Dot_Token), HL);
            Highlight_Name
              (Dotted_Name.F_Suffix, HL, Highlights);
         end;

      when LAL.Ada_Call_Expr =>
         --  Just highlight the name of the called entity
         Highlight_Name
           (LAL.Call_Expr (Name).F_Name, HL, Highlights);

      when others =>
         return;
   end case;
end Highlight_Name;

The above may be quite long, but what it does isn’t new: just as in the Syntax_Highlight function, we execute various actions depending on the kind of the input AST node. If it’s a mere identifier, then we just have to highlight the corresponding only token. If it’s a dotted name (X.Y in Ada), we highlight the prefix (X), the suffix (Y) and the dot in between as names. And so on.

At this point, we could create other syntactic highlighting rules for remaining AST nodes. This blog post is already quite long, so we’ll stop there.

There is one piece that is missing before our syntax highlighter can become actually useful: output formatted source code. Let’s output HTML, as this format is easy to produce and quite universal. We start with a helper analogous to the previous Set procedure, to deal with the dual token/trivia streams:

function Get
  (Highlights : Highlights_Holder;
   Token      : LAL.Token_Data_Type) return Highlight_Type
is
   Index : constant Token_Index := LAL.Index (Token);
begin
   return (if LAL.Is_Trivia (Token)
           then Highlights.Trivia_Highlights (Index)
           else Highlights.Token_Highlights (Index));
end Get;

And now let’s get to the output itself. This starts with a simple iteration on token, so the outline is similar to the first highlighting pass we did above:

Token     : LAL.Token_Type := LAL.First_Token (Unit);
Last_Sloc : Slocs.Source_Location := (1, 1);

Put_Line ("<pre>");
while Token /= LAL.No_Token loop
   declare
      TD         : constant LAL.Token_Data_Type :=
         LAL.Data (Token);
      HL         : constant Highlight_Type :=
         Get (Highlights, TD);
      Sloc_Range : constant Slocs.Source_Location_Range :=
         LAL.Sloc_Range (TD);

      Text : constant Langkit_Support.Text.Text_Type :=
         LAL.Text (Token);
   begin
      while Last_Sloc.Line < Sloc_Range.Start_Line loop
         New_Line;
         Last_Sloc.Line := Last_Sloc.Line + 1;
         Last_Sloc.Column := 1;
      end loop;

      if Sloc_Range.Start_Column > Last_Sloc.Column then
         Indent (Integer (Sloc_Range.Start_Column - Last_Sloc.Column));
      end if;

      Put_Token (Text, HL);
      Last_Sloc := Slocs.End_Sloc (Sloc_Range);
   end;
   Token := LAL.Next (Token);
end loop;
Put_Line ("</pre>");

The tricky part here is that tokens alone are not enough: we use the source location information (line and column numbers) associated to tokens in order to re-create line breaks and whitespaces: this is what the inner while loop and if statement do. As usual, we delegate “low-level” actions to dedicated procedures:

procedure Put_Token
  (Text : Langkit_Support.Text.Text_Type;
   HL   : Highlighter.Highlight_Type) is
begin
   Put ("<span style=""color: #" & Get_HTML_Color (HL)
        & ";"">");
   Put (Escape (Text));
   Put ("</span>");
end Put_Token;

procedure New_Line is
begin
   Put ((1 => ASCII.LF));
end New_Line;

procedure Indent (Length : Natural) is
begin
   Put ((1 .. Length => ' '));
end Indent;

Writing the Escape function, which wraps special HTML characters such as < or > into HTML entities (< and cie), and Get_HTML_Color, which returns a suitable hexadecimal string to encode the color corresponding to a highlighting category (for instance: #ff0000, i.e. the red color, for keywords) is left as an exercise to the reader.

Note that Escape must deal with a Text_Type formal. This type, which is really a subtype of Wide_Wide_String, is used to encode source excerpts in a uniform way in Libadalang, regardless of the input encoding. In order to do something useful with them, one must transcode it to UTF-8, for example. One way to do this is to use GNATCOLL.Iconv, but this is out of the scope of this post.

So here we are! Now you know how to:

  • parse Ada source files with Libadalang;

  • iterate on the stream of tokens/trivia in the resulting analysis unit, as well as process the associated data;

  • traverse the syntax tree of this unit;

  • combine the above in order to create a syntax highlighter.

Thank you for reading this post to the end! If you are interested in pursuing this road, you can find a compilable set of sources for this syntax highlighter on Libadalang’s repository on Github. And because we cannot decently dedicate a whole blog post to a syntax highlighter without a little demo, here is one:

Little demo of Ada source code syntax highlighting with Libadalang
]]>
Pretty-Printing Ada Containers with GDB Scripts http://blog.adacore.com/pretty-printing-ada-containers-with-gdb-scripts Tue, 25 Jul 2017 13:00:00 +0000 Pierre-Marie de Rodat http://blog.adacore.com/pretty-printing-ada-containers-with-gdb-scripts

When things don’t work as expected, developers usually do one of two things: either add debug prints to their programs, or run their programs under a debugger. Today we’ll focus on the latter activity.

Debuggers are fantastic tools. They turn our compiled programs from black boxes into glass ones: you can interrupt your program at any point during its execution, see where it stopped in the source code, inspect the value of your variables (even some specific array item), follow chains of accesses, and even modify all these values live. How powerful! However, sometimes there’s so much information available that navigating through it to reach the bit of state you want to inspect is just too complex.

Take a complex container, such as an ordered map from Ada.Containers.Ordered_Maps, for example. These are implemented in GNAT as  binary trees: collections of nodes, each node having a link to its parent and its left and children a key and a value. Unfortunately, finding a particular node in a debugger that corresponds to the key you are looking for is a painful task. See for yourself:

with Ada.Containers.Ordered_Maps;

procedure PP is
   package Int_To_Nat is
  	new Ada.Containers.Ordered_Maps (Integer, Natural);

   Map : Int_To_Nat.Map;
begin
   for I in 0 .. 9 loop
  	Map.Insert (I, 10 * I);
   end loop;

   Map.Clear;  --  BREAK HERE
end PP;

Build this program with debug information and execute it until line 13:

$ gnatmake -q -g pp.adb
$ gdb -q ./pp
Reading symbols from ./pp...done.
(gdb) break pp.adb:13
Breakpoint 1 at 0x406a81: file pp.adb, line 13.
(gdb) r
Breakpoint 1, pp () at pp.adb:13
13         Map.Clear;  --  BREAK HERE

(gdb) print map
$1 = (tree => (first => 0x64e010, last => 0x64e1c0, root => 0x64e0a0, length => 10, tc => (busy => 0, lock => 0)))

# “map” is a record that contains a bunch of accesses…
# not very helpful. We need to go deeper.
(gdb) print map.tree.first.all
$2 = (parent => 0x64e040, left => 0x0, right => 0x0, color => black, key => 0, element => 0)

# Ok, so what we just saw above is the representation of the node
# that holds the key/value association for key 0 and value 0. This
# first node has no child (see left and right above), so we need to
# inspect its parent:
(gdb) print map.tree.first.parent.all
$3 = (parent => 0x64e0a0, left => 0x64e010, right => 0x64e070, color => black, key => 1, element => 10)

# Great, we havethe second element! It has a left child,
# which is our first node ($2). Now let’s go to its right
# child:
(gdb) print map.tree.first.parent.right.all
$4 = (parent => 0x64e040, left => 0x0, right => 0x0, color => black, key => 2, element => 20)

# That was the third element: this one has no left or right
# child, so we have to get to the parent of $3:
(gdb) print map.tree.first.parent.parent.all
$5 = (parent => 0x0, left => 0x64e040, right => 0x64e100, color => black, key => 3, element => 30)

# So far, so good: we already visited the left child ($4), so
# now we need to visit $5’s right child:
(gdb) print map.tree.first.parent.parent.right.all
$6 = (parent => 0x64e0a0, left => 0x64e0d0, right => 0x64e160, color => black, key => 5, element => 50)

# Key 5? Where’s the node for the key 4? Oh wait, we should
# also visit the left child of $6:
(gdb) print map.tree.first.parent.parent.right.left.all
$7 = (parent => 0x64e100, left => 0x0, right => 0x0, color => black, key => 4, element => 40)

# Ad nauseam…

Manually visiting a binary tree is much easier for computers than it is for humans, as everyone knows. So in this case, it seems easier to write a debug procedure in Ada that iterates on the container and prints each key/value association and call this debug procedure from GDB.

But this has its own drawbacks: first, it forces you to remember to write this procedure for each container instantiation you do, eventually forcing you to rebuild your program each time you debug it. But there’s worse: if you debug your program from a core dump, it’s not possible to call a debug procedure from GDB. Besides, if the state of your program is somehow corrupted, due to stack overflow or a subtle memory handling bug (dangling pointers, etc.), calling this debug procedure will probably corrupt your process even more, making the debugging session a nightmare!

This is where GDB comes to the rescue: there’s a feature called pretty-printers, which makes it possible to hook into GDB to customize how it displays values. For example, you can write a hook that intercepts values whose type matches the instantiation of ordered maps and that displays only the “useful” content of the maps.

We developed several GDB scripts to implement such pretty-printers for the most common standard containers in Ada: not only vectors, hashed/ordered maps/sets, linked lists, but also unbounded strings. You can find them in the dedicated repository hosted on GitHub: https://github.com/AdaCore/gnat-gdb-scripts.

With these scripts properly installed, inspecting the content of containers becomes much easier:

(gdb) print map
$1 = pp.int_to_nat.map of length 10 = {[0] = 0, [1] = 10, [2] = 20,
  [3] = 30, [4] = 40, [5] = 50, [6] = 60, [7] = 70, [8] = 80,
  [9] = 90}

Note that beginning with GNAT Pro 18, GDB ships with these pretty-printers, so there’s no setup other than adding the following commands to your .gdbinit file:

python import gnatdbg; gnatdbg.setup()

Happy debugging!

]]>
The Adaroombot Project http://blog.adacore.com/the-adaroombot-project Tue, 20 Jun 2017 12:48:00 +0000 Rob Tice http://blog.adacore.com/the-adaroombot-project

Owning an iRobot RoombaⓇ is an interesting experience. For those not familiar, the RoombaⓇ is a robot vacuum cleaner that’s about the diameter of a small pizza and stands tall enough to still fit under your bed. It has two independently driven wheels, a small-ish dust bin, a rechargeable battery, and a speaker programmed with pleasant sounding beeps and bloops telling you when it’s starting or stopping a cleaning job. You can set it up to clean on a recurring schedule through buttons on the robot, or with the new models, the mobile app. It picks up an impressive amount of dust and dirt and it makes you wonder how you used to live in a home that was that dirty.

A Project to Learn Ada


I found myself new to AdaCore without any knowledge of the Ada programming language around the same time I acquired a RoombaⓇ for my cats to use as a golf cart when I wasn’t home. In order to really learn Ada I decided I needed a good project to work on. Having come from an embedded Linux C/C++ background I decided to do a project involving a Raspberry Pi and something robotic that it could control. It just so happens that iRobot has a STEM initiative robot called the CreateⓇ 2 which is aimed towards embedded control projects. That’s how the AdaRoombot project was born.

The first goal of the project was to have a simple Ada program use the CreateⓇ 2’s serial port to perform some control algorithm. Mainly this would require the ability to send commands to the robot and receive feedback information from the robot’s numerous sensors. As part of the CreateⓇ 2 documentation package, there is a PDF detailing the serial port interface called the iRobot CreateⓇ 2 Open Interface Specification.

On the command side of things there is a simple protocol: each command starts with a one-byte opcode specifying which command is being issued and then is followed by a number of bytes carrying the data associated with the opcode, or the payload. For example, the Reset command has an opcode of 7 and has zero payload bytes. The Set Day/Time command has an opcode of 168 and has a 3-byte payload with a byte specifying the Day, another for the Hour, and another for the Minute. The interface for the Sensor data is a little more complicated. The host has the ability to request data from individual sensors, a list of sensors, or tell the robot to just stream the list over and over again for processing. To make things simple, I choose to just receive all the sensor data on each request.

Because we are using a Raspberry Pi, it is quite easy to communicate with a serial port using the Linux tty interface. As with most userspace driver interfaces in Linux, you open a file and read and write byte data to the file. So, from a software design perspective, the lowest level of the program abstraction should take robot commands and transform them into byte streams to write to the file, and conversely read bytes from the file and transform the byte data to sensor packets. The next level of the program should perform some algorithm by interpreting sensor data and transmitting commands to make the robot perform some task and the highest level of the program should start and stop the algorithm and do some sort of system monitoring.

The high level control algorithm I used is very simple: drive straight until I hit something, then turn around and repeat. However, the lower levels of the program where I am interfacing with peripherals is much more exciting. In order to talk to the serial port, I needed access to file I/O and Linux’s terminal I/O APIs. 


Ada has cool features

Ada has a nifty way to interface with the Linux C libraries that can be seen near the bottom of “src/communication.ads”. There I am creating Ada specs for C calls, and then telling the compiler to use the C implementations supplied by Linux using pragma Import. This is similar to using extern  in C. I am also using pragma Convention which tells the compiler to treat Ada records like C structs so that they can be passed into the imported C functions. With this I have the ability to interface to any C call I want using Ada, which is pretty cool. Here is an example mapping the C select call into Ada:

--  #include <sys/select.h>
--  fd_set represents file descriptor sets for the select function.
--  It is actually a bit array.
type Fd_Set is mod 2 ** 32;
pragma Convention (C, Fd_Set);

--  #include <sys/time.h>
--  time_t tv_sec - number of whole seconds of elapsed time.
--  long int tv_usec - Rest of the elapsed time in  microseconds.
type Timeval is record
    Tv_Sec  : C.Int;
    Tv_Usec : C.Int;
end record;
pragma Convention (C, Timeval);

function C_Select (Nfds : C.Int;
                  Readfds   : access Fd_Set;
                  Writefds  : access Fd_Set;
                  Exceptfds : access Fd_Set;
                  Timeout   : access Timeval)
                  return C.int;
pragma Import (C, C_Select, "select");

The other neat low-level feature to note here can be seen in “src/types.ads”. The record Sensor_Collection is a description of the data that will be received from the robot over the serial port. I am using a feature called a representation clause to tell the compiler where to put each component of the record in memory, and then overlaying the record on top of a byte stream. By doing this, I don’t have to use any bit masks or bit shift to access individual bits or fields within the byte stream. The compiler has taken care of this for me. Here is an example of a record which consists of Boolean values, or bits in a byte:

type Sensor_Light_Bumper is record
    LT_Bump_Left         : Boolean;
    LT_Bump_Front_Left   : Boolean;
    LT_Bump_Center_Left  : Boolean;
    LT_Bump_Center_Right : Boolean;
    LT_Bump_Front_Right  : Boolean;
    LT_Bump_Right        : Boolean;
end record
 with Size => 8;

for Sensor_Light_Bumper use record
    LT_Bump_Left at 0 range 0 .. 0;
    LT_Bump_Front_Left at 0 range 1 .. 1;
    LT_Bump_Center_Left at 0 range 2 .. 2;
    LT_Bump_Center_Right at 0 range 3 .. 3;
    LT_Bump_Front_Right at 0 range 4 .. 4;
    LT_Bump_Right at 0 range 5 .. 5;
end record;

In this example, LT_Bump_Left is the first bit in the byte, LT_Bump_Front_Left is the next bit, and so on. In order to access these bits, I can simply use the dot notation to access members of the record, where with C I would have to mask and shift. Components that span multiple bytes can also include an endianness specification. This is useful because on this specific platform data is little endian, but the serial port protocol is big endian. So instead of byte swapping, I can specify certain records as having big endian mapping. The compiler then handles the swapping.

These are some of the really cool low level features available in Ada. On the high-level programming side, the algorithm development, Ada feels more like C++, but with some differences in philosophy. For instance, certain design patterns are more cumbersome to implement in Ada because of things like, Ada objects don’t have explicit constructors or destructors. But, after a small change in mind-set it was fairly easy to make the robot drive around the office.

Adaroombot Bottom
Adaroombot Top

The code for AdaRoombot, which is available on Github, can be compiled using the GNAT GPL cross compiler for the Raspberry Pi 2 located on the AdaCore Libre site. The directions to build and run the code is included in the README file in the root directory of the repo. The next step is to add some vision processing and make the robot chase a ball down the hallway. Stay tuned….

The code is available on GitHub: here.


Want to start creating and hacking with Ada? We have a great challenge for you!

The Make with Ada competition, hosted by AdaCore, calls on embedded developers across the globe to build cool embedded applications using the Ada and SPARK programming languages and offers over €8000 in total prizes. Celebrating its sophomore year, the Make With Ada Competition is designed to promote the benefits of the languages to the software development community at large. For more information and to register, go to makewithada.org.

]]>
GNAT GPL 2017 is out! http://blog.adacore.com/gnat-gpl-2017-is-out Thu, 15 Jun 2017 13:20:09 +0000 Pierre-Marie de Rodat http://blog.adacore.com/gnat-gpl-2017-is-out

For those users of the GNAT GPL edition, we are pleased to announce the availability of the 2017 release of GNAT GPL and SPARK GPL.

SPARK GPL 17 offers improved automation of proofs, thanks to improvements in the underlying prover Alt-Ergo and a finer-grain splitting of conjunctions. Interaction during proof has been improved thanks to the new statistics, display and replay modes. Type invariants from Ada are now also supported in SPARK. Note that the optional provers CVC4 and Z3 are no longer distributed with SPARK GPL 2017, and should be installed separately.

This release also marks the first introduction of “future language” Ada 2020 features:

  • delta aggregates (partial aggregate notation)
  • AI12-0150-1 class-wide invariants now employ class-wide pre-/postcondition-like semantics (static call resolution).

This release supports the ARM ELF bare metal target, hosted on Windows and Linux, as well as the following native platforms:

  • Mac OS (64 bits)
  • Linux (64 bits)
  • Windows (32 bits)

The GNATemulator technology has been added to the bare metal target, making it easier to develop and test on those platforms.

The compiler toolchain is now based on GCC 6. The native runtime comes with a Zero Foot Print runtime, and the ARM ELF compiler comes with runtimes for a variety of boards, including support for the Raspberry Pi 2.

The latest version of the GPS IDE comes with many bug fixes and enhancements, notably in the areas of debugger integration and support for bare-metal development.

The GPL 2017 release can be downloaded:

Feeling inspired and want to start Making with Ada today? Check out the Make With Ada Competition! http://makewithada.org

]]>
Ada on the first RISC-V microcontroller http://blog.adacore.com/ada-on-the-first-risc-v-microcontroller Tue, 13 Jun 2017 12:30:00 +0000 Fabien Chouteau http://blog.adacore.com/ada-on-the-first-risc-v-microcontroller

The RISC-V open instruction set is getting more and more news coverage these days. In particular since the release of the first RISC-V microcontroller from SiFive and the announcement of an Arduino board at the Maker Faire Bay Area 2017.

As an Open Source software company we are very interested in this trendy, new, open platform. AdaCore tools already support an open IP core with the Leon processor family, a popular architecture in the space industry that is developed in VHDL and released under the GPL. RISC-V seems to be targeting a different market and opening new horizons.

GNAT - the Ada compiler developed and maintained by AdaCore - is part of the GCC toolchain. As a result, when a new back-end is added we can fairly quickly start targeting it and developing in Ada. In this blog post I will describe the steps I followed to build the tool chain and start programing the HiFive 1 RISC-V microcontroller in Ada.

Building the tool chain

The first step is to build the compiler. SiFive - manufacturer of the MCU - provides an SDK repository with scripts to build a cross RISC-V GCC. All I had to do was to change the configure options to enable Ada support

--enable-languages=c,c++,ada

and disable libada since this is a bare-metal target (no operating system) we won’t use a complete run-time

--disable-libada

If you want to build the toolchain yourself, I forked and modified the freedom-e-sdk repository.

Just clone it

$ git clone --recursive https://github.com/Fabien-Chouteau/freedom-e-sdk

install a native GNAT from your Linux distrib (I use Ubuntu)

$ sudo apt-get install gnat

and start the build

$ cd freedom-e-sdk
$ make tools

If you have a problem with this procedure don’t hesitate to open an issue on GitHub, I’ll see what I can do to help.

Building the run-time

Ada programs always need a run-time library, but there are different run-time profiles depending on the constraints of the platform. In GNAT we have the so called Zero FootPrint run-time (ZFP) that provides the bare minimum and therefore is quite easy to port to a new platform (no exception propagation, no tasking, no containers, no file system access, etc.).

I started from Shawn Nock’s ZFP for the Nordic nRF51 and then simply changed the linker script and startup code, everything else is platform independant.

You can find the run-time in this repository: https://github.com/Fabien-Chouteau/zfp-hifive1

Writing the drivers

To control the board I need a few drivers. I started by writing the description of the hardware registers using the SVD format: here.

I then generated Ada mapping from this file using the SVD2Ada tool. You can find more info about this process at the beginning of this blog post.

From these register mappings it’s fairly easy to implement the drivers. So far I wrote GPIO and UART: https://github.com/AdaCore/Ada_Drivers_Library/tree/master/arch/RISC-V/SiFive/drivers

First Ada projects on the HiFive1

The first project is always a blinky. The HiFive1 has RGB leds so I started by driving those. You can find this example in the Ada_Drivers_Library.

If you want to run this simple example on your board, get my fork of the freedom-e-sdk (as described above) and run:

$ make ada_blinky_build

and then

$ make ada_blinky_upload

to flash the board.

For the second project, thanks to the architecture of the Ada_Drivers_Library, I was able to re-use the thermal printer driver from my DIY instant camera and it took me only 5 minutes to print something from the HiFive1.

Conclusion

All of this is only experimental for the moment, but it shows how quickly we can start programing Ada on new platforms. Proper support would require a run-time with tasking, interruptions, protected objects (Ravenscar profile) and of course complete test and validation of the compiler.


Feeling inspired and want to start Making with Ada today? We have the perfect challenge for you!

The Make with Ada competition, hosted by AdaCore, calls on embedded developers across the globe to build cool embedded applications using the Ada and SPARK programming languages and offers over €8000 in total prizes. Celebrating its sophomore year, the Make With Ada Competition is designed to promote the benefits of the languages to the software.

]]>
DIY Coffee Alarm Clock http://blog.adacore.com/diy-coffee-alarm-clock Tue, 16 May 2017 13:00:00 +0000 Fabien Chouteau http://blog.adacore.com/diy-coffee-alarm-clock

A few weeks ago one of my colleagues shared this kickstarter project : The Barisieur. It’s an alarm clock coffee maker, promising to wake you up with a freshly brewed cup of coffee every morning. I jokingly said “just give me an espresso machine and I can do the same”. Soon after, the coffee machine is in my office. Now it is time to deliver :)

The basic idea is to control the espresso machine from an STM32F469 board and use the beautiful screen to display the clock face and configuration interface.

Hacking the espresso machine

The first step is to be able to control the machine with the 3.3V signal of the microcontroller. To do this, I open the case to get to the two push buttons on the top. Warning! Do not open this kind of appliance if you don’t know what you are doing. First, it can be dangerous, second, these things are not made to be serviceable so there’s a good chance you will never be able to put it back together.

The push buttons are made with two exposed concentric copper traces on a small PCB and a conductive membrane that closes the circuit when the button is pushed.

I use a multimeter to measure the voltage between two circles of one of the buttons. To my surprise the voltage is quite high, about 16V. So I will have to use a MOSFET transistor to act as an electronic switch rather than just connecting the microcontroller to the espresso machine signals.

I put that circuit on an Arduino proto shield that is then plugged behind the STM32F469 disco board. The only things left to do are to drill a hole for the wires to go out of the the machine and to make a couple of metal brackets to attach to the board. Here’s a video showing the entire hacking process.

Writing the alarm clock software

For the clock face and configuration interface I will use Giza, one of my toy projects that I developed to play with the object oriented programming features of Ada. It’s a simplistic/basic UI framework.

Given the resolution of the screen (800x480) and the size of the text I want to display, it will be too slow to use software font rendering. Instead, I will take advantage of the STM32F4’s 2D graphic hardware acceleration (DMA2D) and have some bitmap images embedded in the executable. DMA2D can very quickly copy chunks of memory - typically bitmaps - but also convert them from one format to the other. This project is the opportunity to implement support of indexed bitmap in the Ada_Drivers_Library.

I also add support for STM32F4’s real time clock (RTC) to be able to keep track of time and date and of course trigger the coffee machine at the time configured by the user.

It’s time to put it all together and ask my SO to perform in the the high budget video that you can see at the beginning of this post :)

The code is available on GitHub: here.


Feeling inspired and want to start Making with Ada today? We have the perfect challenge for you!

The Make with Ada competition, hosted by AdaCore, calls on embedded developers across the globe to build cool embedded applications using the Ada and SPARK programming languages and offers over €8000 in total prizes. Celebrating its sophomore year, the Make With Ada Competition is designed to promote the benefits of the languages to the software development community at large. For more information and to register, go to makewithada.org.

]]>
(Many) More Low Hanging Bugs http://blog.adacore.com/many-more-low-hanging-bugs Fri, 05 May 2017 13:00:00 +0000 Yannick Moy http://blog.adacore.com/many-more-low-hanging-bugs

In a previous post, we reported our initial experiments to create lightweight checkers for Ada source code, based on the new Libadalang technology. The two checkers we described discovered 12 issues in the codebase of the tools we develop at AdaCore. In this post, we are reporting on 6 more lightweight checkers, which have discovered 114 new issues in our codebase. These 6 checkers allowed us to detect errors and code quality issues for 4 of them (check_deref_null, check_test_not_null, check_same_test, check_bad_unequal) and refactoring opportunities for 2 of them (check_same_then_else, check_useless_assign). Every checker runs in seconds on our codebase, which made it easy to improve them until the checkers had no false alarms. Currently, none of these checkers uses the recent semantic analysis capability in Libadalang, which might be useful in the future to improve their precision. In each of these checkers, we took inspiration from similar lightweight checkers in other static analysis tools, in particular PVS-Studio and its gallery of real-life examples.

Checkers on Dereference

Our first checker is a favorite of many tools. It checks whether a pointer that has been derefenced, is later tested against the null value. This is suspicious, as we'd expect the sequence of events to be opposite. This can point either to an error (the pointer should not be dereferenced without a null check) or a code quality issue (the null test is useless). In fact, we found both when applying the checker on the codebase of our tools. Here's an example of error found in the GNAT compiler, in g-spipat.adb, where procedure Dump dereferences P.P at line 2088:

   procedure Dump (P : Pattern) is

      subtype Count is Ada.Text_IO.Count;
      Scol : Count;
      --  Used to keep track of column in dump output

      Refs : Ref_Array (1 .. P.P.Index);
      --  We build a reference array whose N'th element points to the
      --  pattern element whose Index value is N.

and then much later at line 2156 it checks for P.P being null:

      --  If uninitialized pattern, dump line and we are done

      if P.P = null then
         Put_Line ("Uninitialized pattern value");
         return;
      end if;

The code was fixed to declare array Refs after we know P.P is not null. And here is an example of code quality issue also in the GNAT compiler, at line 2797 of g-comlin.adb, where parameter Line is dereferenced and then tested against null:

      Sections_List : Argument_List_Access :=
                        new Argument_List'(1 .. 1 => null);
      Found         : Boolean;
      Old_Line      : constant Argument_List := Line.all;
      Old_Sections  : constant Argument_List := Sections.all;
      Old_Params    : constant Argument_List := Params.all;
      Index         : Natural;

   begin
      if Line = null then
         return;
      end if;

The code was fixed by declaring Line as a parameter of a not null access type. In some cases the dereference and the test are both in the same expression, for example in this case in our tool GNATstack, at line 97 of dispatching_calls.adb:

         if
           Static_Class.Derived.Contains (Explored_Method.Vtable_Entry.Class)
              and then
           Explored_Method /= null
              and then

The code was fixed here by checking for non-null before dereferencing Explored_Method. Overall, this checker found 11 errors in our codebase and 9 code quality issues.

A second checker in this category looks for places where a test that a pointer is null dominates a dereference of the same pointer. This is, in general, an indication of a logic error, in particular for complex boolean expressions, as shown by the examples from PVS-Studio gallery. We found no such error in our codebase, which may be an indication of our good test coverage. Indeed, any execution of such code will raise an exception in Ada.

Checkers on Tested Expressions

Our first checker on tested expressions looks for identical subexpressions being tested in a chain of if-elsif statements. It points to either errors or code quality issues. Here is an example of error it found in the GNAT compiler, at line 7380 of sem_ch4.adb:

                  if Nkind (Right_Opnd (N)) = N_Integer_Literal then
                     Remove_Address_Interpretations (Second_Op);

                  elsif Nkind (Right_Opnd (N)) = N_Integer_Literal then
                     Remove_Address_Interpretations (First_Op);
                  end if;

The code was fixed by testing Left_Opnd(N) instead of Right_Opnd(N) in the second test. Overall, this checker found 3 errors in our codebase and 7 code quality issues.

A second checker in this category looks for expressions of the form "A /= B or A /= C" where B and C are different literals, which are always True. In general "and" is meant instead of "or". This checker found one error in our QGen code generator, at line 675 of himoco-blockdiagramcmg.adb:

               if Code_Gen_Mode /= "Function"
                 or else Code_Gen_Mode /= "Reusable function"
               then
                  To_Flatten.Append (Obj);
               end if;

Checkers for Code Duplication

Our first checker for code duplication looks for identical code in different branches of an if-statement or case-statement. It may point to typos or logical errors, but in our codebase it pointed only to refactoring opportunities. Still, some of these cause code duplication of more than 20 lines of code, for example at line 1023 of be-checks.adb in CodePeer:

            elsif VN_Kind (VN) = Binexpr_VN
              and then Operator (VN) = Logical_And_Op
              and then Int_Sets.Is_In (Big_True, To_Int_Set_Part (Expect))
            then
               --  Recurse to propagate check down to operands of "and"
               Do_Check_Sequence
                 (Check_Kind,
                  Split_Logical_Node (First_Operand (VN)),
                  Srcpos,
                  File_Name,
                  First_Operand (VN),
                  Expect,
                  Check_Level,
                  Callee,
                  Callee_VN,
                  Callee_Expect,
                  Callee_Precondition_Index);
               Do_Check_Sequence
                 (Check_Kind,
                  Split_Logical_Node (Second_Operand (VN)),
                  Srcpos,
                  File_Name,
                  Second_Operand (VN),
                  Expect,
                  Check_Level,
                  Callee,
                  Callee_VN,
                  Callee_Expect,
                  Callee_Precondition_Index);
...
            elsif VN_Kind (VN) = Binexpr_VN
              and then Operator (VN) = Logical_Or_Op
              and then Int_Sets.Is_In (Big_False, To_Int_Set_Part (Expect))
            then
               --  Recurse to propagate check down to operands of "and"
               Do_Check_Sequence
                 (Check_Kind,
                  Split_Logical_Node (First_Operand (VN)),
                  Srcpos,
                  File_Name,
                  First_Operand (VN),
                  Expect,
                  Check_Level,
                  Callee,
                  Callee_VN,
                  Callee_Expect,
                  Callee_Precondition_Index);
               Do_Check_Sequence
                 (Check_Kind,
                  Split_Logical_Node (Second_Operand (VN)),
                  Srcpos,
                  File_Name,
                  Second_Operand (VN),
                  Expect,
                  Check_Level,
                  Callee,
                  Callee_VN,
                  Callee_Expect,
                  Callee_Precondition_Index);

or at line 545 of soap-generator-skel.adb in GPRbuild:

                  when WSDL.Types.K_Derived =>

                     if Output.Next = null then
                        Text_IO.Put
                          (Skel_Adb,
                           WSDL.Parameters.To_SOAP
                             (N.all,
                              Object    => "Result",
                              Name      => To_String (N.Name),
                              Type_Name => T_Name));
                     else
                        Text_IO.Put
                          (Skel_Adb,
                           WSDL.Parameters.To_SOAP
                             (N.all,
                              Object    =>
                                "Result."
                                  & Format_Name (O, To_String (N.Name)),
                              Name      => To_String (N.Name),
                              Type_Name => T_Name));
                     end if;

                  when WSDL.Types.K_Enumeration =>

                     if Output.Next = null then
                        Text_IO.Put
                          (Skel_Adb,
                           WSDL.Parameters.To_SOAP
                             (N.all,
                              Object    => "Result",
                              Name      => To_String (N.Name),
                              Type_Name => T_Name));
                     else
                        Text_IO.Put
                          (Skel_Adb,
                           WSDL.Parameters.To_SOAP
                             (N.all,
                              Object    =>
                                "Result."
                                  & Format_Name (O, To_String (N.Name)),
                              Name      => To_String (N.Name),
                              Type_Name => T_Name));
                     end if;

Overall, this checker found 62 code quality issues in our codebase.

Our last checker looks for useless assignment to a local variable, where the value is never read subsequently. This can be very obvious, such as this case at line 1067 of be-value_numbers-factory.adb in CodePeer:

      Global_Obj.Obj_Id_Number    := Obj_Id_Number (New_Obj_Id);
      Global_Obj.Obj_Id_Number    := Obj_Id_Number (New_Obj_Id);

or more hidden, such as this case at line 895 of bt-xml-reader.adb, still in CodePeer:

                              if Next_Info.Sloc.Column = Msg_Loc.Column then
                                 Info := Next_Info;
                                 Elem := Next_Cursor;
                              end if;
                              Elem := Next_Cursor;

Overall, this checker found 9 code quality issues in our codebase.

Setup Recipe 

So you actually want to try the above scripts on your own codebase? This is possible right now with your latest GNAT Pro release or the latest GPL release for community & academic users! Just follow the instructions we described in the Libadalang repository and you will be able to run the scripts inside your favorite Python2 interpreter.

Conclusion

Summing over the 8 checkers that we implemented so far (referenced in this post and a previous one), we've found and fixed 24 errors and 102 code quality issues in our codebase. This is definitely showing that these kind of checkers are worth integrating in static analysis tools and we look forward to integrating these and more in our static analyzer CodePeer for Ada programs.

Another lesson is that all of these checkers were developed in a couple of hours each, thanks to the powerful Python API available with Libadalang. While we had to develop some boilerplate to traverse the AST in various directions and multiple workarounds to make for the absence of semantic analysis in the (then available) version of Libadalang, this was relatively little work, and work that we can expect to share across similar checkers in the future. We're now looking forward to the version of Libadalang with on-demand semantic analysis, which will allow us to create even more powerful and useful checkers.

[cover image by Courtney Lemon]

]]>
A Usable Copy-Paste Detector in A Few Lines of Python http://blog.adacore.com/a-usable-copy-paste-detector-in-few-lines-of-python Tue, 02 May 2017 13:00:00 +0000 Yannick Moy http://blog.adacore.com/a-usable-copy-paste-detector-in-few-lines-of-python

After we created lightweight checkers based on the recent Libadalang technology developed at AdaCore, a colleague gave us the challenge of creating a copy-paste detector based on Libadalang. It turned out to be both easier than anticipated, and much more efficient and effective than we could have hoped for. In the near future, we plan to use this new detector to refactor the codebase of some of our tools.

First Attempt: Hashes and Repeated Suffix Trees

Our naive strategy for detecting copy-paste was to reduce it to a string problem, in order to benefit from the existing efficient algorithms on string problems. Our reasoning was that each line of code could be represented by a hash code, so that a file could be represented by a string of hash codes. After a few Web searches, we found the perfect match for this translated problem, on the WikiPedia page for the longest repeated substring problem, which is helpfully pointing to a C implementation used to solve this problem efficiently based on Suffix Trees, a data structure to represent efficiently all suffixes of a string (say, "adacore", "dacore", "acore", "core", "ore", "re" and "e" if your string is "adacore").

So we came up with an implementation in Python of the copy-paste detector, made up of 3 steps:

Step 1: Transform the source code into a string of hash codes

This a simple traversal of the AST produced by Libadalang, producing roughly a hash for each logical line of code. Traversal is made very easy with the API offered by Libadalang, as each node of the AST is iterable in Python to get its children. For example, here is the default case of the encoding function producing the hash codes:

        # Default case, where we hash the kind of the first token for the node,
        # followed by encodings for its subnodes.
        else:
            return ([Code(hash(node.token_start.kind), node, f)] +
                    list(itertools.chain.from_iterable(
                        [enc(sub) for sub in node])))

We recurse here on the AST to concatenate the substrings of hash codes computed for subnodes. The leaf case is obtained for expressions and simple statements, for which we compute a hash of a string obtained from the list of tokens for the node. The API of Libadalang makes it very easy, using again the ability to iterate over a node to get its children. For example, here is the default case of the function computing the string from a node:

            return ' '.join([node.token_start.kind]
                            + [strcode(sub) for sub in node])

We recurse here on the AST to concatenate the kind of the first token for the node with the substrings computed for subnodes. Of course, we are not interested in exactly representing each line of code in this representation. For example, we represent all identifiers by a special wildcard character $, in order to detect copy-pastes even when identifiers are not the same.

Step 2: Construct the Suffix Tree for the string of hash codes

The algorithm by Ukkonen is quite subtle, but it was easy to translate an existing C implementation into Python. For those curious enough, a very instructive series of 6 blog posts leading to this implementation describes Ukkonen's algorithm in details.

Step 3: Compute the longest repeated substring in the string of hash codes

For that, we look at the internal node of the Suffix Tree constructed above with the greatest height (computed in number of hashes). Indeed, this internal node corresponds to two or more suffixes that share a common prefix. For example, with string "adacore", there is a single internal node, which corresponds to the common prefix "a" for suffixes "adacore" and "acore", after which the suffixes are different. The children of this internal node in the Suffix Tree contain the information of where the suffixes start in the string (position 0 for "adacore" and 2 for "acore"), so we can compute positions in the string of hash codes where hashes are identical and for how many hash codes. Then, we can translate this information into files, lines of code and number of lines.

The steps above allow to detect only the longest copy-paste across a codebase (in terms of number of hash codes, which may be different from number of lines of code). Initially, we did not find a better way to detect all copy-pastes longer than a certain limit than to repeat steps 2 and 3 after we remove from the string of hash codes those that correspond to the copy-paste previously detected. This algorithm ran in about one hour on the full codebase of GPS, consisting in 350 ksloc (as counted by sloccount), and it reported both very valuable copy-pastes of more than 100 lines of code, as well as spurious ones. To be clear, the spurious ones were not bugs in the implementation, but limitations of the algorithm that captured "copy-pastes" that were valid duplications of similar lines of code. Then we improved it.

Improvements: Finer-Grain Encoding and Collapsing

The imprecisions of our initial algorithm came mostly from two sources: it was sometimes ignoring too much of the source code, and sometimes too little. That was the case in particular for the abstraction of all identifiers as the wildcard character $, which led to spurious copy-pastes where the identifiers were semantically meaningful and could not be replaced by any other identifier. We fixed that by distinguishing local identifiers that are abstracted away from global identifiers (from other units) that are preserved, and by preserving all identifiers that could be the names of record components (that is, used in a dot notation like Obj.Component). Another example of too much abstraction was that we abstracted all literals by their kind, which again lead to spurious copy-pastes (think of large aggregates defining the value of constants). We fixed that by preserving the value of literals.

As an example of too little abstraction, we got copy-pastes that consisted mostly of sequences of small 5-to-10 lines subprograms, which could not be refactored usefully to share common code. We fixed that by collapsing sequences of such subprograms into a single hash code, so that their relative importance towards finding large copy-pastes was reduced. We made various other adjustments to the encoding function to modulate the importance of various syntactic constructs, simply by producing more or less hash codes for a given construct. An interesting adjustment consisted in ignoring the closing tokens in a construct (like the "end Proc;" at the end of a procedure) to avoid having copy-pastes that start on such meaningless starting points. It seems to be a typical default of token-based approaches, that our hash-based approach allows to solve easily, by simply not producing a hash for such tokens.

After these various improvements, the analysis of GPS codebase came down to 2 minutes, an impressive improvement from the initial one hour! The code for this version of the copy-paste detector can be found in the GitHub repository of Libadalang.

Optimizations: Suffix Arrays, Single Pass 

To improve on the above running time, we looked for alternative algorithms for performing the same task. And we found one! Suffix Arrays are an alternative to Suffix Trees, which is simpler to implement and from which we saw that we could generate all copy-pastes without regenerating the underlying data structure after finding a given copy-paste. We implemented in Python the algorithm in C++ found in this paper, and the code for this alternative implementation can be found in the GitHub repository of Libadalang. This version found the same copy-pastes as the previous one, as expected, with a running time of 1 minute for the analysis of GPS codebase, a 50% improvement!

Looking more closely at the bridges between Suffix Trees and Suffix Arrays (essentially you can reconstruct one from the other), we also realized that we could use the same one-pass algorithm to detect copy-pastes with Suffix Trees, instead of recreating each time the Suffix Tree for the text where the copy-paste just detected had been removed. The idea is that, instead of repeatedly detecting the longest copy-paste on a newly created Suffix Tree, we traverse the initial Suffix Tree and issue all copy-pastes with a maximal length, where copy-pastes that are not maximal can be easily recognized by checking the previous hash in the candidate suffixes. For example, if two suffixes for this copy-paste start at indexes 5 and 10 in the string of hashes, we check the hashes at indexes 4 and 9: if they are the same, then the copy-paste is not maximal and we do not report it. With this change, the running time for our original algorithm is just above 1 minute for the analysis of GPS codebase, i.e. close to the alternative implementation based on Suffix Arrays.

So we ended up with two implementations for our copy-paste detector, one based on Suffix Trees and one based on Suffix Arrays. We'll need to experiment further to decide which one to keep in a future plug-in for our GPS and GNATbench IDEs.

Results on GPS

The largest source base on which we tried this tool is our IDE GNAT Programming Studio (GPS). This is about 350'000 lines of source code. It uses object orientation, tends to have medium-sized subprograms (20 to 30 lines), although there are some much longer ones. In fact, we aim at reducing the size of the longest subprograms, and a tool like gnatmetric will help find them. We are happy to report that most of the code duplication occurred in recent code, as we are transitioning and rewriting some of the old modules.

Nonetheless, the tool helped detect a number of duplicate chunks, with very few spurious detections (corresponding to cases where the tool reports a copy-paste that turns out to be only similar code).

Let's take a look at three copy-pastes that were detected.

Example 1: Intended temporary duplication of code

gps/gvd/src/debugger-base_gdb-gdb_cli.adb:3267:1: copy-paste of 166 lines detected with code from line 3357 to line 3522 in file gps/gvd/src/debugger-base_gdb-gdb_mi.adb

This is a large subprogram used to handle the Memory view in GPS. We have recently started changing the code to use the gdb MI protocol to communicate with gdb, rather than simulate an interactive session. Since the intent is to remove the old code, the duplication is not so bad, but is useful in reminding us we need to clean things up here, preferably soon before the code diverges too much.

Example 2: Unintended almost duplication of code

gps/builder/core/src/commands-builder-scripts.adb:266:1: copy-paste of 21 lines detected with code from line 289 to line 309

This code is in the handling of the python functions GPS.File.compile() and GPS.File.make(). Interestingly enough, these two functions were not doing the same thing initially, and are also documented differently (make attempts to link the file after compiling it). Yet the code is almost exactly the same, except that GPS does not spawn the same build target (see comment in the code below). So we could definitely use an if-expression here to avoid the duplication of the code.

      elsif Command = "compile" then
         Info := Get_Data (Nth_Arg (Data, 1, Get_File_Class (Kernel)));
         Extra_Args := GNAT.OS_Lib.Argument_String_To_List
           (Nth_Arg (Data, 2, ""));

         Builder := Builder_Context
           (Kernel.Module (Builder_Context_Record'Tag));

         Launch_Target (Builder      => Builder,
                        Target_Name  => Compile_File_Target,    -- <<< use Build_File_Target here for "make"
                        Mode_Name    => "",
                        Force_File   => Info,
                        Extra_Args   => Extra_Args,
                        Quiet        => False,
                        Synchronous  => True,
                        Dialog       => Default,
                        Via_Menu     => False,
                        Background   => False,
                        Main_Project => No_Project,
                        Main         => No_File);

         Free (Extra_Args);

The tool could be slightly more helpful here by highlighting the exact differences between the two blocks. As the blocks get longer, it is harder to spot a change in one identifier (as is the case here). This is where an integration in our IDEs like GPS and GNATbench would be useful, and of course possibly with some support for automatic refactoring of the code, also based on Libadalang.

Example 3: Unintended exact duplication of code

gps/code_analysis/src/codepeer-race_details_models.adb:39:1: copy-paste of 20 lines detected with code from line 41 to line 60 in file gps/code_analysis/src/codepeer-race_summary_models.adb

This one is an exact duplication of a function. The tool could perhaps be slightly more helpful by showing those exact duplicates first, since they will often be the easiest ones to remove, simply by moving the function to the spec.

   function From_Iter (Iter : Gtk.Tree_Model.Gtk_Tree_Iter) return Natural is
      pragma Warnings (Off);
      function To_Integer is
        new Ada.Unchecked_Conversion (System.Address, Integer);
      pragma Warnings (On);

   begin
      if Iter = Gtk.Tree_Model.Null_Iter then
         return 0;

      else
         return To_Integer (Gtk.Tree_Model.Utils.Get_User_Data_1 (Iter));
      end if;
   end From_Iter;

Setup Recipe

So you actually want to try the above scripts on your own codebase? This is possible right now with your latest GNAT Pro release or the latest GPL release for community & academic users! Just follow the instructions we described in the Libadalang repository, you will then be able to run the scripts inside your favorite Python2 interpreter.

Conclusion

What we took from this experiment was that: (1) it is easier than you think to develop a copy-paste detector for your favorite language, and (2) technology like Libadalang is key to facilitate the necessary experiments that lead to an efficient and effective detector. On the algorithmic side, we think it's very beneficial to use a string of hash codes as intermediate representation, as this allows to precisely weigh in which language constructs contribute what.

Interestingly, we did not find other tools or articles describing this type of intermediate approach between token-based approaches and syntactic approaches, which provides an even faster analysis than token-based approaches, while avoiding their typical pitfalls, and allows fine-grained control based on the syntactic structure, without suffering from the long running time typical of syntactic approaches.

We look forward to integrating our copy-paste detector in GPS and GNATbench, obviously initially for Ada, but possibly for other languages as well (for example C and Python) as progress on langkit, the Libadalang's underlying technology, allows. The integration of Libadalang in GPS was completed not long ago, so it's easier than ever.

]]>
GPS for bare-metal developers http://blog.adacore.com/gps-for-bare-metal-development Wed, 19 Apr 2017 12:57:03 +0000 Anthony Leonardo Gracio http://blog.adacore.com/gps-for-bare-metal-development

In my previous blog article, I exposed some techniques that helped me rewrite the Crazyflie’s firmware from C into Ada and SPARK 2014, in order to improve its safety.

I was still an intern at that time and, back in the day, the support for bare-metal development in GPS was a bit minimalistic: daily activities like flashing and debugging my own firmware on the Crazyflie were a bit painful to do without having to go outside of GPS.

This is not the case anymore. GPS now comes with a number of features regarding bare-metal development that make it very easy for newcomers (as I was) to develop their own software for a particular board.

Bare-metal Holy Grail: Build, Flash, Debug

Building your modified software in order to flash it or debug it on a board is a very common workflow in bare-metal development. GPS offers support for all these different steps, allowing you to perform them at once with one single click.

In particular, GPS now supports two different tools for connecting to your remote board in order to flash and/or debug it:

  • ST-Link utility tools (namely st-util and st-flash) for STM32-based boards

  • OpenOCD, a connection tool supporting various types of boards and probes, specifically the ones that use a JTAG interface

Once installed on your host, using these tools in order to flash or debug your project directly from GPS is very easy. As pictures are worth a thousand words, here is a little tutorial video showing how to set up your bare-metal project from GPS in order to build, flash and debug it on a board:



Monitoring the memory usage

When it comes to bare-metal development, flashing and debugging your project on the targeted board is already a pretty advanced step: it means that you have already been able to compile and, above all, to link your software correctly.

The linking phase can be a real pain due to the limited memory resources of these boards: writing software that does not fit in the board’s available memory is something that can happen pretty quickly as long as your project grows and grows.

To address these potential issues, a Memory Usage view has been introduced. By default, this view is automatically spawned each time you build your executable and displays a complete view of the static memory usage consumed by your software, even when the linking phase has failed. The Memory Usage view uses a map file generated by the GNU ld linker to report the memory usage consumed at three different levels:

  1. Memory regions, which correspond to the MEMORY blocks defined in the linker script used to link the executable
  2. Memory sections (e.g: .data, .bss etc.)
  3. Object files

Having a complete report of the memory usage consumed at each level makes it very convenient to identify which parts of your software are consuming too much memory for the available hardware ressources. This is the case in the example below, where we can see that the .bss section of our executable is just too big for the board's RAM, due to the huge uninitialized array declared in leds.ads.

Conclusion

Bare-metal development support has been improved greatly in GPS, making it easier for newcomers to build, flash and debug their software. Moreover, the Memory Usage view allows the bare-metal developers to clearly identify the cause of memory overflows. 

We don't want to stop our efforts regarding bare-metal development support so don't hesitate to submit ideas of improvements on our GitHub's repository!

]]>
User-friendly strings API http://blog.adacore.com/user-friendly-strings-api Mon, 10 Apr 2017 13:47:00 +0000 Emmanuel Briot http://blog.adacore.com/user-friendly-strings-api

User friendly strings API

In a previous post, we described the design of a new strings package, with improved performance compared to the standard Ada unbounded strings implementation. That post focused on various programming techniques used to make that package as fast as possible.

This post is a followup, which describes the design of the user API for the strings package - showing various solutions that can be used in Ada to make user-friendly data structures.

Tagged types

One of the features added in Ada 2005 is a prefix dot notation when calling primitive operations of tagged types. For instance, if you have the following declarations, you can call the subprograms Slice in one of two ways:

declare
   type XString is tagged private;
   procedure Slice (Self : in out XString; Low, High : Integer);   --  a primitive operation
   S : XString;
begin
   S.Slice (1, 2);      --  the two calls do the same thing, here using a prefix notation
   Slice (S, 1, 2);
end;

This is a very minor change. But in practice, people tend to use the prefix notation because it is more "natural" for people who have read or written code in other programming languages.

In fact, it is so popular that there is some demand to extend this prefix notation, in some future version of the language, to types other than tagged types. Using a tagged type has a cost, since it makes variables slightly bigger (they now have a hidden access field).

In our case, though, a XString was already a tagged type since it is also controlled, so there is no additional cost.

Indexing

The standard Ada strings are quite convenient to use (at least once you understand that you should use declare blocks since you need to know their size in advance). For instance, one can access characters with expressions like:

A : constant Character := S (2);      --  assuming 2 is a valid index
B : constant Character := S (S'First + 1);   --  better, of course
C : constant Character := S (S'Last);

The first line of the above example hides one of the difficulties for newcomers: strings can have any index range, so using just "2" is likely to be wrong here. Instead, the second line should be used. The GNATCOLL strings avoid this pitfall by always indexing from 1. As was explained in the first blog post, this is both needed for the code (so that internally we can reallocate strings as needed without changing the indexes manipulated by the user), and more intuitive for a lot of users.

The Ada unbounded string has a similar approach, and all strings are indexed starting at 1. But you can't use the same code as above, and instead you need to write the more cumbersome: 

S := To_Unbounded_String (...);
A : constant Character := Element (S, 2);     --  second character of the string, always
B : constant Character := Ada.Strings.Unbounded.Element (S, 2);   --  when not using use-clauses

GNATCOLL.Strings takes advantage of some new aspects introduced in Ada 2012 to provide custom indexing functions. So the spec looks like:

type XString is tagged private
    with Constant_Indexing  => Get;

function Get (Self : XString; Index : Positive) return Character;

S : XString := ...;
A : constant Character := S (2);    --   always the second character

After adding this simple Constant_Indexing aspect, we are now back to the simple syntax we were using for standard String, using parenthesis to point to a specific character. But here we also know that "2" is always the second character, so we do not need to use the 'First attribute.

Variable indexing

There is a similar aspect, named Variable_Indexing which can be used to modify a specific character of the string, as we would do with a standard string. So we can write:

type XString is tagged private
    with Variable_Indexing  => Reference;

type Character_Reference (Char : not null access Char_Type) is limited null record
    with Implicit_Dereference => Char;

function Reference
    (Self  : aliased in out XString;
     Index : Positive) return Character_Reference;

S : XString := ...;
S (2) := 'A';
S (2).Char := 'A';    --  same as above, but not taking advantage of Implicit_Derefence

Although this is simple to use for users of the library, the implementation is actually much more complex than for Constant_Indexing

First of all, we need to introduce a new type, called a Reference Type (in our example this is the Character_Reference), which basically is a type with an access discriminant and the Implicit_Dereference aspect. This type acts as a safer replacement for an access type (for instance it can't be copied quite as easily). Through this type, we have an access to a specific character, and therefore users can replace that character easily.

But the real difficulty is exactly because of this access. Imaging that we assign the string to another one. At that point, they share the internal buffer when using the copy-on-write mode described in the first post. But if we then modify a character we modify it in both strings, although from the user's point of view these are two different instances ! Here are three examples of code that would fail:


begin
    S2 := S;
    S (2) := 'A';
    --  Now both S2 and S have 'A' as their second character
end;

declare
    R : Character_Reference := S.Reference (2);
begin
    S2 := S;    --   now sharing the internal buffer
    R := 'A';   -- Both S2 and S have 'A' as their second character
end;

declare
    R : Character_Reference := S.Reference (2);
begin
    S.Append ("ABCDEF");
    R := 'A';   --  might point to deallocated memory
end;

Of course, we want our code to be safe, so the implementation needs to prevent the above errors. As soon we take a Reference to a character we must ensure the internal buffer is no longer shared. This fixes the first example above: Since the call to S (2) no longer shares the buffer, we are indeed only modifying S and not S2.

The second example looks similar, but in fact the sharing is done after we have taken the reference. So in fact when we take a Reference we also need to make the internal buffer unshareable. This is done by setting a special value for the refcounting, so that the assignment always make a copy of S's internal buffer, rather than share it.

There is a hidden cost to using Variable_Indexing, since we are no longer able to share the buffer as soon as a reference was taken. This is unfortunately unavoidable, and is one of those examples where convenience of use goes counter to performance...

The third example is much more complex. Here, we have a reference to a character in the string, but when we append data to the string we are likely to reallocate memory. The system could be allocating new memory, copying the data, and then freeing the old one. And our existing reference would point to the memory area that we have just freed !

I have not found a solution here (and C++ implementations seem to have similar limitations). We cannot simply prevent reallocation (i.e. changing the size of the string), since the user might simply have taken a reference, changed the character, and dropped the reference. At that point, reallocating would be safe. For now, we rely on documentation and on the fact that it is some extra work to preserve a reference as we did in the example, it is much more natural to save the index, and then take another reference when needed, as in:


declare
    Saved : constant Integer := 2;
begin
    S.Append ("ABCDEF");
    S (Saved) := 'A';     --  safe
end;

Iteration

Finally, we want to make it easy to traverse a string and read all its characters (and possibly modify them, of course). With the standard strings, one would do:

declare
    S : String (1 .. 10) := ...;
begin
    for Idx in S'Range loop
        S (Idx) := 'A';
    end loop;

    for C of S loop
       Put (C);
    end loop
end;

declare
    S : Unbounded_String := ....;
begin
    for Idx in 1 .. Length (S) loop
       Replace_Element (S, Idx, 'A');
       Put (Element (S, Idx));
    end loop;
end;

Obviously, the version with unbounded strings is a lot more verbose and less user-friendly. It is possible, since Ada 2012, to provide custom iteration for our own strings, via some predefined aspects. As the gem shows, this is a complex operation...

GNAT provides a much simpler aspect that fulfills all practical needs, namely the Iterable aspect. We use it as such:

type XString is tagged private
    with Iterable => (First => First,
                      Next => Next,
                      Has_Element => Has_Element,
                      Get => Get);

function First (Self : XString) return Positive is (1);
function Next (Self : XString; Index : Positive) return Positive is (Index + 1);
function Has_Element (Self : XString; Index : Positive) return Boolean is (Index <= Self.Length);
function Get (Self : Xstring; Idx : Positive) return Character;   --   same one we used for indexing

These functions are especially simple here because we know the range of indexes. So basically they are declared as expression functions and made inline. This means that the loop can be very fast in practice. For consistency with the way the "for...of" loop is used with standard Ada containers, we chose to have Get return the character itself, rather than its index.  And now we can use:

S : XString := ...;

for C of S loop
    Put (C);
end loop;

As explained, the for..of loop returns the character itself, which means we can't change the string directly. So we need to provide a second iterator that will return the indexes. We do this via a new small type that takes care of everything, as in:

 type Index_Range is record
    Low, High : Natural;
 end record
 with Iterable => (First       => First,
                   Next        => Next,
                   Has_Element => Has_Element,
                   Element     => Element);

 function First (Self : Index_Range) return Positive is (Self.Low);
 function Next (Self : Index_Range; Index : Positive) return Positive  is (Index + 1);
 function Has_Element (Self : Index_Range; Index : Positive)  return Boolean is (Index <= Self.High);
 function Element
    (Self : Index_Range; Index : Positive) return Positive
    is (Index);

function Iterate (Self : XString) return Index_Range
    is ((Low => 1, High => Self.Length));

S : XString := ...;

for Idx in S.Iterate loop
    S (Idx) := 'A';
end loop;

The type Index_Range is very similar to the previous use of the Iterable aspect. We just need to introduce one extra primitive operation for the string, which is used to initiate the iteration.

In fact, we could make Iterate and Index_Range more complex, so that for instance we can iterate on every other character, or on every 'A' in the string, or any kind of iteration scheme. This would of course require a more complex Index_Range, but opens up nice possibilities.

Slicing

With standard strings, one can get a slice by using notation like:

  declare
      S : String (1 .. 10) := ...;
   begin
      Put (S (2 .. 5));
      S (2 .. 5) := 'ABCD';
   end;

Unfortunately, there is no possible equivalent for GNATCOLL strings, because the range notation ("..") cannot be redefined. We have to use code like:

declare
    S : XString := ...;
begin
    Put (S.Slice (2, 5));
    S.Replace (2, 5, 'ABCD');
end;

It would be nice if Ada had a new aspect to let us map the ".." notation to a function, for instance something like:

type XString is tagged private
    with  Range_Indexing  => Slice,           --   NOT Ada 2012
          Variable_Range_Indexing => Replace;    --   NOT Ada 2012

function Slice (Self : XString; Low, High : Integer) return XString;
--  Low and High must be of same type, but that type could be anything
--  The return value could also be anything

procedure Replace (S : in out XString; Low, High : Integer; Replace_With : String);
--  Likewise, Low and High must be of same type, but this could be anything.
--  and Replace_With could be any type

Perhaps in some future version of the language?

Conclusion

Ada 2012 provides quite a number of new aspects that can be applied to types, and make user API more convenient to use. Some of them are complex to use, and GNAT sometimes provides a simpler version.

]]>
New strings package in GNATCOLL http://blog.adacore.com/new-strings-package-in-gnatcoll Tue, 04 Apr 2017 13:30:00 +0000 Emmanuel Briot http://blog.adacore.com/new-strings-package-in-gnatcoll

New strings package in GNATCOLL

GNATCOLL has recently acquired two new packages, namely GNATCOLL.Strings and GNATCOLL.Strings_Impl. The latter is a generic package, one instance of which is provided as GNATCOLL.Strings.

But why a new strings package? Ada already has quite a lot of ways to represent and manipulate strings, is a new one needed?

This new package is an attempt at finding a middle ground between the standard String type (which is efficient but inflexible) and unbounded strings (which are flexible, but could be more efficient).

GNATCOLL.Strings therefore provides strings (named XString, as in extended-strings) that can grow as needed (up to Natural'Last, like standard strings), yet are faster than unbounded strings. They also come with an extended API, which includes all primitive operations from unbounded strings, in addition to some subprograms inspired from GNATCOLL.Utils and the Python and C++ programming languages.

Small string optimization

GNATCOLL.Strings uses a number of tricks to improve on the efficiency.  The most important one is to limit the number of memory allocations.  For this, we use a trick similar to what all C++ implementations do nowadays, namely the small string optimization.

The idea is that when a string is short, we can avoid all memory allocations altogether, while still keeping the string type itself small. We therefore use an Unchecked_Union, where a string can be viewed in two ways:

    Small string

      [f][s][ characters of the string 23 bytes               ]
         f  = 1 bit for a flag, set to 0 for a small string
         s  = 7 bits for the size of the string (i.e. number of significant
              characters in the array)

    Big string

      [f][c      ][size     ][data      ][first     ][pad    ]
         f = 1 bit for a flag, set to 1 for a big string
         c = 31 bits for half the capacity. This is the size of the buffer
             pointed to by data, and which contains the actual characters of
             the string.
         size = 32 bits for the size of the string, i.e. the number of
             significant characters in the buffer.
         data = a pointer (32 or 64 bits depending on architecture)
         first = 32 bits, see the handling of substrings below
         pad = 32 bits on a 64 bits system, 0 otherwise.
             This is because of alignment issues.

So in the same amount of memory (24 bytes), we can either store a small string of 23 characters or less with no memory allocations, or a big string that requires allocation. In a typical application, most strings are smaller than 23 bytes, so we are saving very significant time here.

This representation has to work on both 32 bits systems and 64 bits systems, so we have careful representation clauses to take this into account.  It also needs to work on both big-endian and little-endian systems. Thanks to Ada's representation clauses, this one in fact relatively easy to achieve (well, okay, after trying a few different approaches to emulate what's done in C++, and that did not work elegantly). In fact, emulating via bit-shift operations ended up with code that was less efficient than letting the compiler do it automatically because of our representation clauses.

Character types

Applications should be able to handle the whole set of unicode characters. In Ada, these are represented as the Wide_Character type, rather than Character, and stored on 2 bytes rather than 1. Of course, for a lot of applications it would be wasting memory to always store 2 bytes per character, so we want to give flexibility to users here.

So the package GNATCOLL.Strings_Impl is a generic. It has several formal parameters, among which:

   * Character_Type is the type used to represent each character. Typically,  it will be Character, Wide_Character, or even possibly Wide_Wide_Character. It could really be any scalar type, so for instance we could use this package to represent DNA with its 4-valued nucleobases.

   * Character_String is an array of these characters, as would be represented in Ada. It will typically be a String or a Wide_String. This type is used to make this package work with the rest of the Ada world.

Note about unicode: we could also always use a Character, and use UTF-8 encoding internally. But this makes all operations (from taking the length to moving the next character) slower, and more fragile. We must make sure not to cut a string in the middle of a multi-byte sequence. Instead, we manipulate a string of code points (in terms of unicode). A similar choice is made in Ada (String vs Wide_String), Python and C++.

Configuring the size of small strings

The above is what is done for most C++ implementations nowadays.  The maximum 23 characters we mentioned for a small string depends in fact on several criteria, which impact the actual maximum size of a small string:

   * on 32 bits system, the size of the big string is 16 bytes, so the maximum size of a small string is 15 bytes.
   * on 64 bits system, the size of the big string is 24 bytes, so the maximum size of a small string is 23 bytes.
   * If using a Character as the character type, the above are the actual number of characters in the string. But if you are using a Wide_Character, this is double the maximum length of the string, so a small string is either 7 characters or 11 characters long.

This is often a reasonable number, and given that applications mostly use small strings, we are already saving a lot of allocations. However, in some cases we know that the typical length of strings in a particular context is different. For instance, GNATCOLL.Traces builds messages to output in the log file. Such messages will typically be at most 100 characters, although they can of course be much larger sometimes.

We have added one more formal parameter to GNATCOLL.Strings_Impl to control the maximum size of small strings. If for instance we decide that a "small" string is anywhere from 1 to 100 characters long (i.e. we do not want to allocate memory for those strings), it can be done via this parameter. Of course, in such cases the size of the string itself becomes much larger.

In this example it would be 101 bytes long, rather than the 24 bytes.  Although we are saving on memory allocations, we are also spending more time copying data when the string is passed around, so you'll need to measure the performance here.

The maximum size for the small string is 127 bytes however, because this size and the 1-bit flag need to fit in 1 bytes in the representation clauses we showed above. We tried to make this more configurable, but this makes things significantly more complex between little-endian and big-endian systems, and having large "small" strings would not make much sense in terms of performance anyway.

Typical C++ implementations do not make this small size configurable.

Task safety

Just like unbounded strings, the strings in this package are not thread safe. This means that you cannot access the same string (read or write) from two different threads without somehow protecting the access via a protected type, locks,...

In practice, sharing strings would rarely be done, so if the package itself was doing its own locking we would end up with very bad performance in all cases, for a few cases where it might prove useful.

As we'll discuss below, it is possible to use two different strings that actually share the same internal buffer, from two different threads. Since this is an implementation detail, this package takes care of guaranteeing the integrity of the shared data in such a case.

Copy on write

There is one more formal parameter, to configure whether this package should use copy-on-write or not. When copy on write is enabled, you can have multiple strings that internally share the same buffer of characters. This means that assigning a string to another one becomes a reasonably fast operation (copy a pointer and increment a refcount). Whenever the string is modified, a copy of the buffer is done so that other copies of the same string are not impacted.

But in fact, there is one drawback with this scheme: we need reference counting to know when we can free the shared data, or when we need to make a copy of it. This reference counting must be thread safe, since users might be using two different strings from two different threads, but they share data internally.

Thus the reference counting is done via atomic operations, which have some impact on performance. Since multiple threads try to access the same memory addresses, this is also a source of contention in multi-threaded applications.

For this reason, the current C++ standard prevents the use of copy-on-write for strings.

In our cases, we chose to make this configurable in the generic, so that users can decide whether to pay the cost of the atomic operations, but save on the number of memory allocations and copy of the characters.  Sometimes it is better to share the data, but sometimes it is better to systematically copy it. Again, actual measurements of the performance are needed for your specific application.

Growth strategy

When the current size of the string becomes bigger than the available allocated memory (for instance because you are appending characters), this package needs to reallocate memory. There are plenty of strategies here, from allocating only the exact amount of memory needed (which saves on memory usage, but is very bad in terms of performance), to doubling the current size of the string until we have enough space, as currently done in the GNAT unbounded strings implementation.

The latter approach would therefore allocate space for two characters, then for 4, then 8 and so on.

This package has a slightly different strategy. Remember that we only start allocating memory past the size of small strings, so we will for instance first allocate 24 bytes. When more memory is needed, we multiply this size by 1.5, which some researchers have found to be a good comprise between waste of memory and number of allocations. For very large strings, we always allocate multiples of the memory page size (4096 bytes), since this is what the system will make available anyway. So we will basically allocate the following: 24, 36, 54, 82, 122,...

An additional constraint is that we only ever allocate even number of bytes. This is called the capacity of the string. In the layout of the big string, as shown above, we store half that capacity, which saves one bit that we use for the flag.

Growing memory

This package does not use the Ada new operator. Instead, we use functions from System.Memory directly, in part so that we can use the realloc system call. This is much more efficient when we need to grow the internal buffer, since in most cases we won't have to copy the characters at all. Saving on those additional copies has a significant impact, as we'll see in the performance measurements below.

Substrings

One other optimization performed by this package (which is not done for unbounded strings or various C++ implementations) is to optimize substrings when also using copy-on-write.

We simply store the index within the shared buffer of the first character of the string , instead of always starting at one.

From the user's point of view, this is an implementation detail. Strings are always indexed from 1, and internally we convert to an actual position in the buffer. This means that if we need to reallocate the buffer, for instance when the string is modified, we transparently change the index of the first character, but the indexes the user was using are still valid.

This results in very significant savings, as shown below in the timings for Trim for instance. Also, we can do an operation like splitting a string very efficiently.

For instance, the following code doesn't allocate any memory, beside setting the initial value of the string. It parses a file containing some "key=value" lines, with optional spaces, and possibly empty lines:



     declare
        S, Key, Value : XString;
        L             : XString_Array (1 .. 2);
        Last          : Natural;
     begin
        S.Set (".......");

        --  Get each line
        for Line in S.Split (ASCII.LF) loop

           --  Split into at most two substrings
           Line.Split ('=', Into => L, Last => Last);

           if Last = 2 then
              Key := L (1);
              Key.Trim;    --  Removing leading and trailing spaces

              Value := L (2);
              Value.Trim;

           end if;
        end loop;
     end;

Conclusion

We use various tricks to improve the performance over unbounded strings.  From not allocating any memory when a string is 24 bytes (or a configurable limit) or less, to growing the string as needed via a careful growth strategy, to sharing internal data until we need to modify it, these various techniques combine to provide a fast and flexible implementation.

Here are some timing experiments done on a laptop (multiple operating systems lead to similar results). The exact timings are irrelevant, so the results are given in percentage of what the unbounded string takes to performance similar operations.

We configured the GNATCOLL.Strings package in different ways, either with or without copy-on-write, and with a small string size from the default 0..23 bytes or a larger 0..127 bytes.

    Setting a small string multiple times (e.g. Set_Unbounded_String)

        unbounded_string                 = 100 %
        xstring-23 without copy on write =  11 %
        xstring-23 with copy on write    =  12 %
        xstring-127 with copy on write   =  17 %

        Here we see that not doing any memory allocation makes XString
        much faster than unbounded string. Most of the time is spent
        copying characters around, via memcpy.

    Setting a large string multiple times (e.g. Set_Unbounded_String)

        unbounded_string                 = 100 %
        xstring-23 without copy on write =  41 %
        xstring-23 with copy on write    =  50 %
        xstring-127 with copy on write   =  32 %

        Here, XString apparently proves better are reusing already
        allocated memory, although the timings are similar when creating
        new strings instead. Most of the difference is probably related to the
        use of realloc instead of alloc.

    Assigning small strings (e.g.   S2 := S1)

        unbounded_string                 = 100 %
        xstring-23 without copy on write =  31 %
        xstring-23 with copy on write    =  27 %
        xstring-127 with copy on write   =  57 %

    Assigning large strings (e.g.   S2 := S1)

        unbounded_string                 = 100 %
        xstring-23 without copy on write = 299 %
        xstring-23 with copy on write    =  63 %
        xstring-127 with copy on write   =  60 %

        When not using copy-on-write (which unbounded strings do), we need
        to reallocate memory, which shows on the second line.

    Appending to large string  (e.g.    Append (S, "...."))

        unbounded_string                 = 100 %
        xstring-23 without copy on write =  39 %
        xstring-23 with copy on write    =  48 %
            same, with tasking           = 142 %
        xstring-127 with copy on write   =  49 %

        When we use tasking, XStrings use atomic operations for the reference
        counter, which slows things down. They become slower than unbounded
        strings, because the latter in fact have a bug when using two
        different strings from two different threads, and they share data
        (they try to save on an atomic operation... this bug is being worked
        on).

    Removing leading and trailing spaces (e.g.   Trim (S, Ada.Strings.Both))

        unbounded_string                 = 100 %
        xstring-23 without copy on write =  50 %
        xstring-23 with copy on write    =  16 %
        xstring-127 with copy on write   =  18 %

        Here we see the benefits of the substrings optimization, which shares
        data for the substrings.
]]>
Simics helps run 60 000 GNAT Pro tests in 24 hours http://blog.adacore.com/simics-helps-run-60-000-gnat-pro-tests-in-24-hours Fri, 31 Mar 2017 14:06:00 +0000 Jerome Guitton http://blog.adacore.com/simics-helps-run-60-000-gnat-pro-tests-in-24-hours

This post has been updated in March 2017 and was originally posted in March 2016.

A key aspect of AdaCore’s GNAT Pro offering is the quality of the product we’re delivering and our proactive approach to resolving issues when they appear. To do so, we need both intensive testing before delivering anything to our customers and to produce “wavefront” versions every day for each product we offer. Doing so each and every day is a real challenge, considering the number of supported configurations, the number of tests to run, and the limit of a 24-hour timeframe. At AdaCore, we rely heavily on virtualization as part of our testing strategy. In this article, we will describe the extent of our GNAT Pro testing on VxWorks, and how Simics helped us meet these challenges.

Broad Support for VxWorks Products

The number of tests to run is proportional to the number of configurations that we support. We have an impressive matrix of configurations to validate:

  • Versions: VxWorks, VxWorks Cert, and VxWorks 653… available on the full range of versions that we support (e.g. 5.5, 6.4 to 6.9, 653 2.1 to 2.5...)
  • CPUs: arm, ppc, e500v2, and x86...
  • Program type: Real Time Process, Downloadable Kernel Module, Static Kernel Module, vThreads, ARINC 653 processes, and with the Cert subsets…
  • Ada configuration variants: zero cost exceptions versus setjmp/longjmp exceptions and Ravenscar tasking profile versus full Ada tasking...

Naturally, there are some combinations in this matrix of possibility that are not supported by GNAT, but the reality is we cover most of it. So the variety of configurations is very high.

The matrix is growing fast. Between 2013 and 2015, we have widened our offer to support the new VxWorks ports (VxWorks 7, VxWorks 653 3.0.x) on a large range of CPUs (arm, e500v2, ppc..), including new configurations that were needed by GNAT Pro users (x86_64). This represents 32 new supported configurations to be added to the 168 existing ones. It was obviously a challenge to qualify all these new versions against our existing test suites, with the goal of supporting the new VxWorks versions as soon as possible.

Simics supports a wide range of VxWorks configurations, and has a good integration with the core OS itself. This made Simics a natural solution for our testing strategy of GNAT Pro. On VxWorks 7 and 653 3.0.x, it allowed us to quickly set up an appropriate testing environment, as predefined material exists to make it work smoothly with most Wind River OSes; we could focus on our own technology right away, instead of spending time on developing, stabilizing and maintaining a new testing infrastructure from scratch.

Representative Testing on Virtual Hardware

Another benefit of Simics is that it not only supports all versions of VxWorks, but also emulates a wide range of hardware platforms. This allows us to test on platforms which are representative of the hardware that will be used in production by GNAT Pro users.

A stable framework for QA led to productivity increases

Stability is an important property of the testing framework; otherwise, “glitches” caused by the testing framework start causing spurious failures, often randomly, that the QA team then needs to analyze each time. Multiplied by the very large number of tests we run each day, and the large number of platforms we test on, lack of stability can quickly lead to an unmanageable situation. Trust of the test framework is a key factor for efficiency.

In that respect, and despite the heavy parallelism required in order to complete the validation of all our products in time, Simics proved to be a robust solution and behaved well under pressure, thus helping us focus our energy on differences caused by our tools, rather than on spurious differences caused by the testing framework.

Test execution speed allowed extensive testing

Some additional info about how broad our testing is. On VxWorks, we mostly have three sets of testsuites:

  • The standard Ada validation testsuite (ACATS): around 3,600 tests;
  • The regression testing of GNAT: around 15,000 tests;
  • Tool-specific testsuites: around 1,800 tests.

All in all, just counting the VxWorks platforms, we are running around 350,000 tests each day. 60,000 of them are run on Simics, mostly on the more recent ports (VxWorks 7 and 653 3.x).

In order to run all these tests, an efficient testing platform is needed. With Simics, we were able to optimize the execution by:

  • Proper tuning of the simulation target;
  • Stopping and restarting the platform at an execution point where tests can be run right away, using checkpoints;
  • Developing additional plugins to have efficient access to the host filesystem from the simulator.

We will give more technical details about these optimizations in a future article.

March 2017 Update

AdaCore’s use of Simics has not substantially changed since last year: it is still a key component of GNAT Pro’s testing strategy on VxWorks platforms. In 2017, GNAT Pro has been ported to PowerPC 64 VxWorks 7; with its broad support for VxWorks products, Simics has been a natural solution to speed up this port.

]]>
GNATcoverage moves to GitHub http://blog.adacore.com/gnatcoverage-moves-to-github Wed, 29 Mar 2017 13:13:51 +0000 Pierre-Marie de Rodat http://blog.adacore.com/gnatcoverage-moves-to-github

Following the current trend, the GNATcoverage project moves to GitHub! Our new address is: https://github.com/AdaCore/gnatcoverage

GNATcoverage is a tool we developed to analyze and report program coverage. It supports the Ada and C programming languages, several native and embedded platforms, as well as various coverage criteria, from object code level instruction and branch coverage up to source level decision or MC/DC coverage, qualified for use in avionics certification contexts. For source-level analysis, GNATcoverage works hand-in-hand with the GNAT Pro compilers.

Originally developed as part of the Couverture research project, GNATcoverage became a supported product for a first set of targets in the 2009/2010 timeframe.

Since the beginning of the project, the development happened on the OpenDO forge. This has served us well, but we are now in the process of moving all our projects to GitHub. What does this change for you?

  • We will now use GitHub issues and pull requests for discussions that previously happened on mailing lists. We hope these places will be more visible and community-friendly.

In the near future, we’ll close the project on the OpenDO forge. We are keen to see you on GitHub!

]]>
Writing on Air http://blog.adacore.com/writing-on-air Mon, 27 Mar 2017 13:00:00 +0000 Jorge Real http://blog.adacore.com/writing-on-air

While searching for motivating projects for students of the Real-Time Systems course here at Universitat Politècnica de València, we found a curious device that produces a fascinating effect. It holds a 12 cm bar from its bottom and makes it swing, like an upside-down pendulum, at a frequency of nearly 9 Hz. The free end of the bar holds a row of eight LEDs. With careful and timely switching of those LEDs, and due to visual persistence, it creates the illusion of text... floating in the air!

The web shows plenty of references to different realizations of this idea. They are typically used for displaying date, time, and also rudimentary graphics. Try searching for "pendulum clock LED", for example. The picture in Figure 1 shows the one we are using.

Figure 1. The pendulum, speaking about itself

The software behind this toy is a motivating case for the students, and it contains enough real-time and synchronisation requirements to also make it challenging. 

We have equipped the lab with a set of these pendulums, from which we have disabled all the control electronics and replaced them with STM32F4 Discovery boards. We use also a self-made interface board (behind the Discovery board in Figure 1) to connect the Discovery with the LEDs and other relevant signals of the pendulum. The task we propose our students is to make it work under the control of a Ravenscar program running on the Discovery. We use GNAT GPL 2016 for ARM hosted on Linux, along with the Ada Drivers Library.

There are two different problems to solve: one is to make the pendulum bar oscillate with a regular period; the other one is to then use the LEDs to display some text.

Swing that bar!

The bar is fixed from the bottom to a flexible metal plate (see Figure 2). The stable position of the pendulum is vertical and still. There is a permanent magnet attached to the pendulum, so that the solenoid behind it can be energised to cause a repulsion force that makes the bar start to swing.

Figure 2. Pendulum mechanics
Figure 3. Detail of barrier pass detector

At startup, the solenoid control is completely blind to the whereabouts of the pendulum. An initial sequence must be programmed with the purpose of moving the bar enough to make it cross the barrier (see detail in Figure 3), a pass detector that uses an opto-coupler sensor located slightly off-center the pendulum run. This asymmetry is crucial, as we'll soon see.

Once the bar crosses the barrier at least three times, we have an idea about the pendulum position along time and we can then apply a more precise control sequence to keep the pendulum swinging regularly. The situation is pretty much like swinging a kid swing: you need to give it a small, regular push, at the right time. In our case, that time is when the pendulum enters the solenoid area on its way to the right side, since the solenoid repels the pendulum rightwards. That happens at about one sixth of the pendulum cycle, so we first need to know when the cycle starts and what duration it has. And for that, we need to pay close attention to the only input of the pendulum: the barrier signal.

Figure 4 sketches a chronogram of the barrier signal. Due to its asymmetric placement, the signal captured from the opto-coupler is also asymmetric.

Figure 4. Chronogram of the barrier signal and correspondence with extreme pendulum positions

To determine the start time and period of the next cycle, we take note of the times when rising and falling edges of the barrier signal occur. This is easy work for a small Finite State Machine (FSM), triggered by barrier interrupts to the Discovery board. Once we have collected the five edge times T1 to T5 (normally would correspond to 2 full barrier crossings plus the start of a third one) we can calculate the period by subtracting T5 - T1. Regarding the start time of the next cycle, we know the pendulum initiated a new cycle (reached its left-most position) just in between the two closest pulses (times T1 and T4). So, based on the information gathered, we estimate that the next cycle will start at time T5 + (T4 - T1) / 2.

But… all we know when we detect a barrier edge is whether it is rising or falling. So, when we detect the first rising edge of Barrier, we can't be sure whether it corresponds to T1 (the second barrier crossing) or T3 (the first). We have arbitrarily guessed it is T1, so we must verify this guess and fix things if it was incorrect. This check is possible precisely due to the asymmetric placement of the pass detector: if our guess was correct, then T3 - T1 should be less than T5 - T3. Otherwise we need to re-assign our measurements (T3, T4 and T5 become T1, T2 and T3) and then move on to the adequate FSM state (waiting for T4).

Once we know when the pendulum will be in the left-most position (the cycle start time) and the estimated duration of the next cycle, we can give a solenoid pulse at the cycle start time plus one sixth of the period. The pulse duration, within reasonable range, affects mostly the amplitude of the pendulum run, but not so much its period. Experiments with pulse durations between 15 and 38 milliseconds showed visible changes in amplitude, but period variations of only about 100 microseconds, for a period of 115 milliseconds (less than 0.1%). We found 18-20 ms to work well.

So, are we done with the pendulum control? Well... almost, but no, we’re not: we are also concerned by robustness. The software must be prepared for unexpected situations, such as someone or something suddenly stopping the bar. If our program ultimately relies on barrier interrupts and they do not occur, then it is bound to hang. A timeout timing event is an ideal mechanism to revive a dying pendulum. If the timeout expires, then the barrier-based control is abandoned and the initialisation phase engaged again, and again if needed, until the pendulum makes sufficient barrier crossings to let the program retake normal operation. After adding this recovery mechanism, we can say we are done with the pendulum control: the bar will keep on swinging while powered.

Adding lyrics to that swing

Once the pendulum is moving at a stable rate, we are ready to tackle the second part of the project: using the eight LEDs to display some text. Knowing the cycle start time and estimated period duration, one can devise a plan to display each line of a character at the proper period times. We have already calculated the next cycle start time and duration for the pendulum control. All we need to do now is to timely provide that information to a displaying task.

Figure 5. Time to display an exclamation mark!

The pendulum control functions described above are implemented by a package with the following (abbreviated) specification:

        with STM32F4;       use STM32F4;
        with Ada.Real_Time; use Ada.Real_Time;

        package Pendulum_IO is

           --  Set LEDs using byte pattern (1 => On, 0 => Off)
           procedure Set_LEDs (Pattern : in Byte);  

           --  Synchronization point with start of new cycle
           procedure Wait_For_Next_Cycle (Init_Time      : out Time; 
                                          Cycle_Duration : out Time_Span);

        private
              task P_Controller with Storage_Size => 4 * 1024;
        end Pendulum_IO;

The specification includes subprograms for setting the LEDs (only one variant shown here) and procedure Wait_For_Next_Cycle, which in turn calls a protected entry whose barrier (in the Ada sense, this time) is opened by the barrier signal interrupt handler, when the next cycle timing is known. This happens at time T5 (see Figure 4), when the current cycle is about to end but with sufficient time before the calling task must start switching LEDs. The P_Controller task in the private part is the one in charge of keeping the pendulum oscillating.

Upon completion of a call to Wait_For_Next_Cycle, the caller knows the start time and period of the next pendulum cycle (parameters Init_Time and Cycle_Period). By division of the period, we can also determine at what precise times we need to switch the LEDs. Each character is encoded using an 8 tall x 5 wide dot matrix, and we want to fit 14 characters in the display. Adding some left and right margins to avoid the slowest segments, and a blank space to the right of each character, we  subdivide the period in 208 lines. These lines represent time windows to display each particular character chunk. Since the pendulum period is around 115 milliseconds, it takes just some 550 microseconds for the pendulum to traverse one line.

If that seems tight, there is an even tighter requirement than this inter-line delay. The LEDs must be switched on only during an interval between 100 and 200 microseconds. Otherwise we would see segments, rather than dots, due to the pendulum speed. This must also be taken into account when designing the plan for the period, because the strategy changes slightly depending on the current pendulum direction. When it moves from left to right, the first 100 microseconds of a line correspond to it's left part, whereas the opposite is true for the opposite direction.


Dancing with the runtime

Apart from careful planning of the sequence to switch the LEDs, this part is possibly less complex, due to the help of Wait_For_Next_Cycle. However, the short delays imposed by the pendulum have put us in front of a limitation of the runtime support. The first try to display some text was far from satisfactory. Often times, dots became segments. Visual glitches happened all the time as well. Following the track to this issue, we ended up digging into the Ravenscar runtime (the full version included in GNAT GPL 2016) to eventually find that the programmed precision for timing events and delay statements was set to one millisecond. This setting may be fine for less demanding applications, and it causes a relatively low runtime overhead; but it was making it impossible for us to operate within the pendulum’s tight delays. Things started to go well after we modified and recompiled the runtime sources to make delays and timing events accurate to 10 microseconds. It was just a constant declaration, but it was not trivial to find it! Definitely, this is not a problem we ask our students to solve: they use the modified runtime.

If you come across the same issue and the default accuracy of 1 millisecond is insufficient for your application, look for the declaration of constant Tick_Period in the body of package System.BB.Board_Support (file s-bbbosu.adb in the gnarl-common part of either the full or the small footprint versions of the runtime). For an accuracy of 10 microseconds, we set the constant to Clock_Frequency / 100_000.

More fun

There are many other things that can be done with the pendulum, such as scrolling a text longer than the display width, or varying the scrolling speed by pressing the user button in the Discovery board (both features are illustrated in the video below, best viewed in HD); or varying the intensity of the text by changing the LEDs flashing time; or displaying graphics rather than just text... 

One of the uses we have given the pendulum is as a chronometer display for times such as the pendulum period, the solenoid pulse width, or other internal program delays. This use has proved very helpful to better understand the process at hand and also to diagnose the runtime delay accuracy issue. 

The pendulum can also be used as a rudimentary oscilloscope. Figure 6 shows the pendulum drawing the chronograms of the barrier signal and the solenoid pulse. The top two lines represent these signals, respectively, as the pendulum moves rightwards. The two bottom lines are for the leftwards semi-period and must be read leftwards. In Figure 7, the two semi-periods are chronologically re-arranged. The result cannot be read as in a real oscilloscope, because of the varying pendulum speed; but knowing that, it is indicative.

Figure 6. Pendulum used as an oscilloscope (original image)
Figure 7. Oscilloscope image, chronologically re-arranged

Want to see it?

I plan to take a pendulum with me to the Ada-Europe 2017 Conference in Vienna. It will be on display during the central days of the conference (13, 14 and 15 June) and I'll be around for questions and suggestions.

Credit, where it's due

My colleague Ismael Ripoll was the one who called my attention to the pendulum, back in 2005. We implemented the text part only (we did not disconnect the solenoid from the original pendulum's microcontroller). Until porting (and extending) this project to the Discovery board, we’ve been using an industrial PC with a digital I/O card to display text in the pendulum. The current setup is about two orders of magnitude cheaper. And it also fits much better the new focus of the subject on real-time and also embedded systems.

I'm thankful to Vicente Lliso, technician at the DISCA department of UPV, for the design and implementation of the adapter card connecting the Discovery board with the pendulum, for his valuable comments and for patiently attending my requests for small changes here and there.

My friend and amateur photographer Álvaro Doménech produced excellent photographical material to decorate this entry, as well as the pendulum video. Álvaro is however not to be blamed for the oscilloscope pictures, which I took with just a mobile phone camera.

And many thanks to Pat Rogers, from AdaCore, who helped me with the delay accuracy issue and invited me to write this entry. It was one of Pat's excellent tutorials at the Ada-Europe conference that pushed me into giving a new angle to this pendulum project... and to others in the near future!

]]>
SPARK Tetris on the Arduboy http://blog.adacore.com/spark-tetris-on-the-arduboy Mon, 20 Mar 2017 13:00:00 +0000 Fabien Chouteau http://blog.adacore.com/spark-tetris-on-the-arduboy

One of us got hooked on the promise of a credit-card-size programmable pocket game under the name of Arduboy and participated in its kickstarter in 2015. The kickstarter was successful (but late) and delivered  the expected working board in mid 2016. Of course, the idea from the start was to program it in Ada , but this is an 8-bits AVR microcontroller (the ATmega32u4 by Atmel) not supported anymore by GNAT Pro. One solution would have been to rebuild our own GNAT compiler for 8-bit AVR from the GNAT FSF repository and use the AVR-Ada project. Another solution, which we explore in this blog post, is to use the SPARK-to-C compiler that we developed at AdaCore to turn our Ada code into C and then use the Arduino toolchain to compile for the Arduboy board.

This is in fact a solution we are now proposing to those who need to compile their code for a target where we do not propose an Ada compiler, in particular small microcontrollers used in industrial automation and automotive industries. Thanks to SPARK-to-C, you can now develop your code in SPARK, compile it to C, and finally compile the generated C code to your target. We have built the universal SPARK compiler! This product will be available to AdaCore customers in the coming months.

We started from the version of Tetris in SPARK that we already ported to the Atmel SAM4S, Pebble-Time smartwatch and Unity game engine. For the details on what is proved on Tetris, see the recording of a talk at FOSDEM 2017 conference. The goal was to make this program run on the Arduboy.

SPARK-to-C Compiler

What we call the SPARK-to-C compiler in fact accepts both less and more than SPARK language as input. It allows pointers (which are not allowed in SPARK) but rejects tagged types and tasking (which are allowed in SPARK). The reason this is the case is that it’s easy to compile Ada pointers into C pointers but much harder to support object oriented or concurrent programming.

SPARK-to-C supports, in particular, all of Ada’s scalar types (enumerations, integers, floating-point, fixed-point, and access) as well as records and arrays and subtypes of these. More importantly, it can generate all the run-time checks to detect violations of type constraints such as integer and float range checks and checks for array accesses out of bounds and access to a null pointer or invalid pointer. Therefore, you can program in Ada and get the guarantee that the executable compiled from the C code generated by SPARK-to-C preserves the integrity of the program, as if you had compiled it directly from Ada with GNAT.

Compiling Ada into C poses interesting challenges. Some of them are resolved by following the same strategy used by GNAT during compilation to binary code. For example, bounds of unconstrained arrays are bundled with the data for the array in so-called "fat pointers", so that both code that directly references Array'First and Array'Last as well as runtime checks for array accesses can access the array bounds in C. This is also how exceptions, both explicit in the code and generated for runtime checks, are handled. Raising an exception is translated into a call to the so-called "last chance handler", a function provided by the user that can perform some logging before terminating the program. This is exactly how exceptions are handled in Ada for targets that don’t have runtime support. In general, SPARK-to-C provides very little runtime support, mostly for numerical computations (sin, cosine, etc.), accessing a real time clock, and outputting characters and strings. Other features require specific source-to-source transformations of Ada programs. For example, functions that return arrays in Ada are transformed into procedures with an additional output parameter (a pointer to some preallocated space in the caller) in C.

The most complex part of SPARK-to-C deals with unnesting nested subprograms because, while GCC supports nested functions as an extension, this is not part of standard C. Hence C compilers cannot be expected to deal with nested functions. Unnesting in SPARK-to-C relies on a tight integration of a source-to-source transformation of Ada code in the GNAT frontend, with special handling of nested subprograms in the C-generation backend. Essentially, the GNAT frontend creates an 'activation record' that contains a pointer field for each uplevel variable referenced in the nested subprogram. The nested subprogram is then transformed to reference uplevel variables through the pointers in the activation record passed as additional parameters. A further difficulty is making this work for indirect references to uplevel variables and through references to uplevel types based on these variables (for example the bound of an array type). SPARK-to-C deals also with these cases: you can find all details in the comments of the compiler file exp_unst.ads

Compiling Tetris from SPARK to C

Once SPARK-to-C is installed, the code of Tetris can be compiled into C with the version of GPRbuild that ships i SPARK-to-C:

$ gprbuild -P<project> --target=c

For example, the SPARK expression function Is_Empty from Tetris code:

function Is_Empty (B : Board; Y : Integer; X : Integer) return Boolean is
      (X in X_Coord and then Y in Y_Coord and then B(Y)(X) = Empty);

is compiled into the C function tetris_functional__is_empty, with explicit checking of array bounds before accessing the board:

boolean tetris_functional__is_empty(tetris_functional__board b, integer y, integer x) {
  boolean C123s = false;
  if ((x >= 1 && x <= 10) && (y >= 1 && y <= 50)) {
    if (!((integer)y >= 1 && (integer)y <= 50))
      __gnat_last_chance_handler(NULL, 0);
    if (!((integer)x >= 1 && (integer)x <= 10))
      __gnat_last_chance_handler(NULL, 0);
    if ((b)[y - 1][x - 1] == tetris_functional__empty) {
      C123s = true;
    }
  }
  return (C123s);
}

or into the following simpler C function when using compilation switch -gnatp to avoid runtime checking:

boolean tetris_functional__is_empty(tetris_functional__board b, integer y, integer x) {
  return (((x >= 1 && x <= 10) && (y >= 1 && y <= 50)) && ((b)[y - 1][x - 1] == tetris_functional__empty));
}

SPARK to C Tetris

Running on Arduboy

To interface the SPARK Tetris implementation with the C API of the Arduboy, we use the standard language interfacing method of SPARK/Ada:

procedure Arduboy_Set_Screen_Pixel (X : Integer; Y : Integer);
pragma Import (C, Arduboy_Set_Screen_Pixel, "set_screen_pixel");

A procedure Arduboy_Set_Screen_Pixel is declared in Ada but not implemented. The pragma Import tells the compiler that this procedure is implemented in C with the name “set_screen_pixel”.

SPARK-to-C will translate calls to the procedure “Arduboy_Set_Screen_Pixel” to calls to the C function “set_screen_pixel”. We use the same technique for all the subprograms that are required for the game (button_right_pressed, clear_screen, game_over, etc.).

The program entry point is in the Arduino sketch file SPARK_Tetris_Arduboy.ino (link). In this file, we define and export the C functions (set_screen_pixel() for instance) and call the SPARK/Ada code with _ada_main_tetris().

It’s that simple :)

If you have an Arduboy, you can try this demo by first following the quick start guide, downloading the project from GitHub, loading the Arduino sketch SPARK_Tetris_Arduboy/SPARK_Tetris_Arduboy.ino, and then clicking the upload button.

]]>
AdaCore attends FOSDEM http://blog.adacore.com/adacore-attends-fosdem Wed, 22 Feb 2017 05:00:00 +0000 AdaCore Admin http://blog.adacore.com/adacore-attends-fosdem

Earlier this month AdaCore attended FOSDEM in Brussels, an event focused on the use of free and open source software. Two members of our technical team were there to give talks: "Prove with SPARK: No Math, Just Code" from Yannick Moy and "64 bit Bare Metal Programming on RPI-3" from Tristan Gingold.

Yannick Moy presented how to prove key properties of Tetris in SPARK and run it on ARM Cortex M. The presentation focused on the accessibility of the proof technology to software engineers, who simply have to code the specification, which does not require a specific math background. Click here to watch Yannick's presentation in full.

Tristan Gingold presented the Raspberry PI 3 board, how to write and build a first example and a demo of a more advanced multi-core application. With almost no tutorials on the internet, he addressed the main new feature of the popular RPI-3 board: 4x 64 bit cores. Click here to watch Tristan's presentation in full. 

For more information and updates on future events that AdaCore will be attending, please visit our website or follow our Twitter @AdaCoreCompany.

]]>
Getting started with the Ada Drivers Library device drivers http://blog.adacore.com/getting-started-with-the-ada-drivers-library-device-drivers Tue, 14 Feb 2017 14:00:00 +0000 Pat Rogers http://blog.adacore.com/getting-started-with-the-ada-drivers-library-device-drivers

The Ada Drivers Library (ADL) is a collection of Ada device drivers and examples for ARM-based embedded targets. The library is maintained by AdaCore, with development originally (and predominantly) by AdaCore personnel but also by the Ada community at large.  It is available on GitHub and is licensed for both proprietary and non-proprietary use.

The ADL includes high-level examples in a directory at the top of the library hierarchy. These examples employ a number of independent components such as cameras, displays, and touch screens, as well as middleware services and other device-independent interfaces. The stand-alone components are independent of any given target platform and appear in numerous products. (A previous blog entry examined one such component, the Bosch BLO055 inertial measurement unit (IMU)). Other examples show how to create high-level abstractions from low-level devices. For instance, one shows how to create abstract data types representing serial ports.

In this entry we want to highlight another extremely useful resource: demonstrations for the low-level device drivers. Most of these drivers are for devices located within the MCU package itself, such as GPIO, UART/USART, DMA, ADC and DAC, and timers. Other demonstrations are for some of the stand-alone components that are included in the supported target boards, for example gyroscopes and accelerometers. Still other demonstrations are for vendor-defined hardware such as a random number generator.

These demonstrations show a specific utilization of a device, or in some cases, a combination of devices. As such they do not have the same purpose as the high-level examples. They may just display values on an LCD screen or blink LEDs. Their purpose is to provide working examples that can be used as starting points when incorporating devices into client applications. As working driver API references they are invaluable.

Approach

Typically there are multiple, independent demonstration projects for each device driver because each is intended to show a specific utilization. For example, there are five distinct demonstrations for the analog-to-digital conversion (ADC) driver. One shows how to set up the driver to use polling to get the converted value. Another shows how to configure the driver to use interrupts instead of polling. Yet another shows using a timer to trigger the conversions, and another builds on that to show the use of DMA to get the converted values to the user. In each case we simply display the resulting values on an LCD screen rather than using them in some larger application-oriented sense.

Some drivers, the I2C and SPI communication drivers specifically,  do not have dedicated demonstrations of their own. They are used to implement drivers for devices that use those protocols, i.e., the drivers for the stand-alone components. The Bosch BLO055 IMU mentioned earlier is an example.

Some of the demonstrations illustrate vendor-specific capabilities beyond typical functionality. The STM32 timers, for example, have direct support for quadrature motor encoders. This support provides CPU-free detection of motor rotation to a resolution of a fraction of a degree. Once the timer is configured for this purpose the application merely samples a register to get the encoder count. The timer will even provide the rotation direction. See the encoder demonstration if interested.

Implementation

All of the drivers and demonstration programs are written in Ada 2012. They use preconditions and postconditions, especially when the driver is complicated. The preconditions capture API usage requirements that are otherwise expressed only within the documentation, and sometimes not expressed at all. Similarly, postconditions help clients understand the effects of calls to the API routines, effects that are, again, only expressed in the documentation. Some of the devices are highly sophisticated -- a nice way of saying blindingly complicated -- and their documentation is complicated too. Preconditions and postconditions provide an ideal means of capturing information from the documentation, along with overall driver usage experience. The postconditions also help with the driver implementation itself, acting as unit tests to ensure implementer understanding. Other Ada 2012 features are also used, e.g., conditional and quantified expressions.

The STM32.Timers package uses preconditions and postconditions extensively because the STM timers are "highly sophisticated." STM provides several kinds of timer with significantly different capabilities. Some are defined as "basic," some "advanced," and others are "general purpose." The only way to know which is which is by the timer naming scheme ("TIM" followed by a number) and the documentation.  Hence TIM1 and TIM8 are advanced timers, whereas TIM6 and TIM7 are basic timers.  TIM2 through TIM5 are general purpose timers but not the same as TIM9 through TIM14, which are also general purpose. We use preconditions and postconditions to help keep it all straight. For example, here is the declaration of the routine for enabling an interrupt on a given timer. There are several timer interrupts possible, represented by the enumeration type Timer_Interrupt. The issue is that basic timers can only have one of the possible interrupts specified, and only advanced timers can have two of those possible. The preconditions express those restrictions to clients.

procedure Enable_Interrupt
   (This   : in out Timer;
    Source : Timer_Interrupt)
with  
   Pre =>
      (if Basic_Timer (This) then Source = Timer_Update_Interrupt) and
      (if Source in Timer_COM_Interrupt | Timer_Break_Interrupt then Advanced_Timer (This)),
   Post => Interrupt_Enabled (This, Source);

The preconditions reference Boolean functions Basic_Timer and Advanced_Timer in order to distinguish among the categories of timers. They simply compare the timer specified to a list of timers in those categories. 

The postcondition tells us that the interrupt will be enabled after the call returns. That is useful for the user but also for the implementer because it serves as an actual check that the implementation does what is expected. When working with hardware, though, we have to keep in mind that the hardware may clear the tested condition before the postcondition code is called. For example, a routine may set a bit in a register in order to make the attached device do something, but the device may clear the bit as part of its response. That would likely happen before the postcondition code could check that the bit was set. When looking throughout the drivers code you may notice some "obvious" postconditions are not specified. That may be the cause.

The drivers use compiler-dependent facilities only when essential. In particular, they use an AdaCore-defined aspect specifying that access to a given memory-mapped register is atomic even when only one part of it is read or updated. This access reflects the hardware requirements and simplifies the driver implementation code considerably.

Organization

The device driver demonstrations are vendor-specific because the corresponding devices exist either within the vendor-defined MCU packages or outside the MCU on the vendors' target boards. The first vendor supported by the library was STMicroelectroncs (STM), although other vendors are beginning to be represented too. As a result, the device driver demonstrations are currently for MCU products and boards from STM and are, therefore, located in a library subdirectory specific to STM. Look for them in the /Ada_Drivers_Library/ARM/STM32/driver_demos/ subdirectory of your local copy from GitHub. There you will see some drivers immediately. These are for drivers that are shared across an entire MCU family. Others are located in further subdirectories containing either a unique device's driver, or devices that do exist across multiple MCUs but nonetheless differ in some significant way.

Let's look at one of the demonstration projects, the "demo_LIS3DSH_tilt" project, so that we can highlight the more important parts.  This program demonstrates basic use of the LIS3DSH accelerometer chip. The four LEDs surrounding the accelerometer will come on and off as the board is moved, reflecting the directions of the accelerations measured by the device.

The first thing to notice is the "readme.md" file. As you might guess, this file explains what the project demonstrates and, if necessary, how to set it up. In this particular case the text also mentions the board that is intended for execution, albeit implicitly, because the text mentions the four LEDs and an accelerometer that are specific to one of the STM32 Discovery boards. In other words, the demo is intended for a specific target board. At the time of this writing, all the STM demonstration projects run on either the STM32F4 or the STM32F429I Discovery boards from STMicroelectronics. They are very inexpensive, amazingly powerful boards. Some demonstrations will run on either one because they do not use board-specific resources.

But even if a demonstration does not require a specific target board, it still matters which board you use because the demo's project file (the "gpr file") specifies the target. If you use a different target the executable will download but may not run correctly, perhaps not at all.

The executable may not run because the specified target's runtime library is used to build the binary executable. These libraries have configurations that reflect the differences in the target board, especially memory and clock rates, so using the runtime that matches the board is critical. This is the first thing to check when the board you are using simply won't run the demonstration at all.

The demonstration project file specifies the target by naming another project in a with-clause. This other project represents a specific target board. Here is the elided content of this demonstration's project file. Note the second with-clause that specifies a gpr file for the STM32F407 Discovery board. That is one of the two lines to change if you want to use the F429I Discovery instead.

with "../../../../boards/common_config.gpr";
with "../../../../boards/stm32f407_discovery.gpr";

project Demo_LIS3DSH_Tilt extends "../../../../examples/common/common.gpr" is

   ...
   for Runtime ("Ada") use STM32F407_Discovery'Runtime("Ada");
   ...

end Demo_LIS3DSH_Tilt;

The other line to change in the project file is the one specifying the "Runtime" attribute. Note how the the value of the attribute is specified in terms of another project's Runtime attribute. That other project is the one named in the second with-clause, so when we change the with-clause we must change the name of the referenced project too.

That's really all you need to change in the gpr file. GPS and the builder will handle everything else automatically.

There is, however, another effect of the with-clause naming a specific target. The demonstration programs must refer to the target MCU in order to use the devices in the MCU package. They may also need to refer to devices on the target board. Different MCU packages have differing numbers of devices (eg, USARTs) in the package. Similarly, different boards have different external components (accelerometers versus gyroscopes, for example). We don't want to limit the code in the ADL to work with specific boards, but that would be the case if the code referenced the targets by name, via packages representing the specific MCUs and boards. Therefore, the ADL defines two packages that represent the MCU and the board indirectly. These are the STM32.Device and STM32.Board packages, respectively. The indirection is then resolved by the gpr file named in the with-clause. In this demonstration the clause names the STM32F407_Discovery project so that is the target board represented by the STM32.Board package. That board uses an STM32F407VG MCU so that is the MCU represented by the STM32.Device package. Each package contains declarations for objects and constants representing the specific devices on that specific MCU and target board.

You'll also see a file named ".gdbinit" at the same level as the readme.md and gpr files. This is a local gdb script that automatically resets the board when debugging. It is convenient but not essential.

At that same level you'll also see a "gnat.adc" file containing configuration pragmas. These files contain a single pragma that ensures all interrupt handlers are elaborated before any interrupts can trigger them, among other things. It is not essential for the correct function of these demonstrations but is a good idea in general.

Other than those files you'll see subdirectories for the source files and compiler's products (the object and ALI files, and the executable file).

And that's it. Invoke GPS on the project file and everything will be handled by the IDE.

Application Use

We mentioned that you must change the gpr file if you want to use a different target board. That assumes you are running the demonstration programs themselves. There is no requirement that you do so. You could certainly take the bulk of the code and use it on some other target that has the same MCU family inside. That's the whole point of the demonstrations: showing how to use the device drivers! The Certyflie project, also on the AdaCore GitHub, is just such a project. It uses these device drivers so it uses an STM32.Device package for the on-board STM32F405 MCU, but the target board is a quad-copter instead of one of the Discovery kits.

Concluding Remarks

Finally, it must be said that not all available devices have drivers in the ADL, although the most important do. More drivers and demonstrations are needed. For example, the hash processor and the cryptographic processor on the STM Cortex-M4 MCUs do not yet have drivers. Other important drivers are missing as well. CAN and Ethernet support is either minimal or lacking entirely. And that's not even mentioning the other vendors possible. We need the active participation of the Ada community and hope you will join us!

]]>
Going After the Low Hanging Bug http://blog.adacore.com/going-after-the-low-hanging-bug Mon, 30 Jan 2017 14:00:00 +0000 Raphaël Amiard http://blog.adacore.com/going-after-the-low-hanging-bug

At AdaCore, we've been developing deep static analysis tools (CodePeer and SPARK) since 2008. And if you factor in the fact that some developers of these tools had been developing these tools (or others) for the past decades, it's fair to say that we've got a deep expertise in deep static analysis tools.

At the same time, many Web companies have adopted light-weight static analysis integrated in their agile code-review-commit cycle. Some of these tools are deployed for checking all commits in the huge codebases of Google (Tricorder) or Facebook (Infer). Others are commercial tools implementing hundreds of small checkers (SonarLintPVS-Studio, Flawfinder, PC-lint, CppCheck). The GNAT compiler implements some of these checkers through its warnings, our coding standard checker GNATcheck implements others, but we are also missing in our technology some useful checkers that rely on intraprocedural or interprocedural control and data flow analysis typically out of reach of a compiler or a coding standard checker.

Some of these checkers are in fact implemented with much greater precision in our deep static analysis tools CodePeer and SPARK, but running these tools requires developing a degree of expertise in the underlying technology to be used effectively, and their use can be costly in terms of resources (machines, people). Hence, these tools are typically used for high assurance software, where the additional confidence provided by deep static analysis outweights the costs. In addition, our tools target mostly absence of run-time errors and a few logical errors like unused variables or statements and a few suspicious constructs. Thus they don't cover the full spectrum of checkers implemented in light-weight static analyzers.

Luckily, the recent Libadalang technology, developed at AdaCore, provides an ideal basis on which to develop such light-weight static analysis, as it can parse and analyze thousands of lines of code in seconds. As an experiment, we implemented two simple checkers using the Python binding of Libadalang, and we found a dozen bugs in the codebases of the tools we develop at AdaCore (including the compiler and static analyzers). That's what we describe in the following.


Checker 1: When Computing is a Waste of Cycles

The first checker detects arguments of some arithmetic and comparison operators which are syntactically identical, in cases where this could be expressed with a constant instead (like "X - X" or "X <= X"):

import libadalang as lal

def same_tokens(left, right):
    return len(left) == len(right) and all(
        le.kind == ri.kind and le.text == ri.text
        for le, ri in zip(left, right)
    )

def has_same_operands(binop):
    return same_tokens(list(binop.f_left.tokens), list(binop.f_right.tokens))

def interesting_oper(op):
    return not isinstance(op, (lal.OpMult, lal.OpPlus, lal.OpDoubleDot,
                               lal.OpPow, lal.OpConcat))

c = lal.AnalysisContext('utf-8')
unit = c.get_from_file(source_file)
for b in unit.root.findall(lal.BinOp):
    if interesting_oper(b.f_op) and has_same_operands(b):
        print 'Same operands for {} in {}'.format(b, source_file)

(Full source available at https://github.com/AdaCore/lib...)

Despite all the extensive testing that is done on our products, this simple 20-lines checker found 1 bug in the GNAT compiler, 3 bugs in CodePeer static analyzer and 1 bug in the GPS IDE! We show them below so that you can convince yourself that they are true bugs, really worth finding.

The bug in GNAT is on the following code in sem_prag.adb:

            --  Attribute 'Result matches attribute 'Result

            elsif Is_Attribute_Result (Dep_Item)
              and then Is_Attribute_Result (Dep_Item)
            then
               Matched := True;

One or the references to Dep_Item should really be Ref_Item. Here is the correct version:

            --  Attribute 'Result matches attribute 'Result

            elsif Is_Attribute_Result (Dep_Item)
              and then Is_Attribute_Result (Ref_Item)
            then
               Matched := True;

Similarly, one of the three bugs in CodePeer can be found in be-ssa-value_numbering-stacks.adb:

            --  Recurse on array base and sliding amounts;
            if VN_Kind (Addr_With_Index) = Sliding_Address_VN
              and then Num_Sliding_Amounts (Addr_With_Index) =
                         Num_Sliding_Amounts (Addr_With_Index)
            then

where one of the references to Addr_With_Index above should really be to Addr_With_Others_VN, and the other two are in be-value_numbers.adb:

         return VN_Global_Obj_Id (VN2).Obj_Id_Number =
           VN_Global_Obj_Id (VN2).Obj_Id_Number
           and then VN_Global_Obj_Id (VN2).Enclosing_Module =
           VN_Global_Obj_Id (VN2).Enclosing_Module;

where two of the four references to VN2 should really be to VN1.

The bug in GPS is in language-tree-database.adb:

               if Get_Construct (Old_Obj).Attributes /=
                 Get_Construct (New_Obj).Attributes
                 or else Get_Construct (Old_Obj).Is_Declaration /=
                 Get_Construct (New_Obj).Is_Declaration
                 or else Get_Construct (Old_Obj).Visibility /=
                 Get_Construct (Old_Obj).Visibility

The last reference to Old_Obj should really be New_Obj.


Checker 2: When Testing Gives No Information

The second checker detects syntactically identical expressions which are chained together in a chain of logical operators, so that one of the two identical tests is useless (as in "A or B or A"):

import libadalang as lal

def list_operands(binop):
    def list_sub_operands(expr, op):
        if isinstance(expr, lal.BinOp) and type(expr.f_op) is type(op):
            return (list_sub_operands(expr.f_left, op)
                    + list_sub_operands(expr.f_right, op))
        else:
            return [expr]

    op = binop.f_op
    return (list_sub_operands(binop.f_left, op)
            + list_sub_operands(binop.f_right, op))

def is_bool_literal(expr):
    return (isinstance(expr, lal.Identifier)
            and expr.text.lower() in ['true', 'false'])

def has_same_operands(expr):
    ops = set()
    for op in list_operands(expr):
        tokens = tuple((t.kind, t.text) for t in op.tokens)
        if tokens in ops:
            return op
        ops.add(tokens)

def same_as_parent(binop):
    par = binop.parent
    return (isinstance(binop, lal.BinOp)
            and isinstance(par, lal.BinOp)
            and type(binop.f_op) is type(par.f_op))

def interesting_oper(op):
    return isinstance(op, (lal.OpAnd, lal.OpOr, lal.OpAndThen, lal.OpOrElse,
                           lal.OpXor))

c = lal.AnalysisContext('utf-8')
unit = c.get_from_file(source_file)
for b in unit.root.findall(lambda e: isinstance(e, lal.BinOp)):
    if interesting_oper(b.f_op) and not same_as_parent(b):
        oper = has_same_operands(b)
        if oper:
            print 'Same operand {} for {} in {}'.format(oper, b, source_file)

(Full source available at https://github.com/AdaCore/lib...)

Again, this simple 40-lines checker found 4 code quality issues in the GNAT compiler, 2 bugs in CodePeer static analyzer, 1 bug and 1 code quality issue in GPS IDE and 1 bug in QGen code generator. Ouch!

The four code quality issues in GNAT are simply duplicated checks that are not useful. For example in par-endh.adb:

         --  Cases of normal tokens following an END

          (Token = Tok_Case   or else
           Token = Tok_For    or else
           Token = Tok_If     or else
           Token = Tok_Loop   or else
           Token = Tok_Record or else
           Token = Tok_Select or else

         --  Cases of bogus keywords ending loops

           Token = Tok_For    or else
           Token = Tok_While  or else

The test "Token = Tok_For" is present twice. Probably better for maintenance to have it once only. The three other issues are similar.

The two bugs in CodePeer are in utils-arithmetic-set_arithmetic.adb:

      Result  : constant Boolean
        := (not Is_Singleton_Set (Set1)) and then (not Is_Singleton_Set (Set1))
        and then (Num_Range_Pairs (Set1, Set2) > Range_Pair_Limit);

The second occurrence of Set1 should really be Set2.

The code quality issue in GPS is in mi-parser.adb:

           Token = "traceframe-changed" or else
           Token = "traceframe-changed" or else

The last line is useless. The bug in GPS is in vcs.adb:

      return (S1.Label = null and then S2.Label = null
              and then S2.Icon_Name = null and then S2.Icon_Name = null)

The first reference to S2 on the last line should really be S1. Note that this issue had already been detected by CodePeer, which is run as part of GPS quality assurance, and it had been fixed on the trunk by one of the GPS developers. Interestingly here, two tools using either a syntactic heuristic or a deep semantic interpretation (allowing CodePeer to detect that "S2.Icon_Name = null" is always true when reaching the last subexpression) reach the same conclusion on that code.

Finally, the bug in QGen is in himoco-change_buffers.ads:

   procedure Apply_Update (Self : in out Change_Buffer)
   with Post =>
   --  @req TR-CHB-Apply_Update
   --  All signals, blocks and variables to move shall
   --  be moved into their new container. All signals to merge shall be
   --  merged.
     (
        (for all Elem of Self.Signals_To_Move =>
             (Elem.S.Container.Equals (Elem.Move_Into.all)))
      and then
        (for all Elem of Self.Blocks_To_Move =>
             (if Elem.B.Container.Is_Not_Null then
                  Elem.B.Container.Equals (Elem.Move_Into.all)))
      and then
        (for all Elem of Self.Signals_To_Move =>
             (Elem.S.Container.Equals (Elem.Move_Into.all)))
      and then

The first and third conjuncts in the postcondition are the same. After checking with QGen developers, the final check here was actually meant for Self.Variables_To_Move instead of Self.Signals_To_Move. So we detected here a bug in the specification (expressed as a contract), using a simple syntactic checker!


Setup Recipe

So you actually want to try the above scripts on your own codebase? This is possible right now with your latest GNAT Pro release or the latest GPL release for community & academic users! Just follow the instructions we described in the Libadalang repository, you will then be able to run the scripts inside your favorite Python2 interpreter.


Conclusion

Overall, this little experiment was eye opening, in particular for us who develop these tools, as we did not expect such gross mistakes to have gone through our rigorous reviews and testing. We will continue investigating what benefits light-weight static analysis might provide, and if this investigation is successful, we will certainly include this capability in our tools. Stay tuned!

[cover image by Max Pixel, Creative Commons Zero - CC0]

]]>
Introducing Libadalang http://blog.adacore.com/introducing-libadalang Mon, 23 Jan 2017 14:00:00 +0000 Raphaël Amiard http://blog.adacore.com/introducing-libadalang

Show me your abstract syntax tree

AdaCore is working on a host of tools that works on Ada code. The compiler, GNAT, is the most famous and prominent one, but it is far from being the only one.

Over the years we have done tools with a variety of requirements regarding processing Ada code, that we can put on a spectrum:

  • Some tools, like a compiler, or some static analyzers, will need to ensure that they are working on correct Ada code, both from a syntactic and semantic point of view.

  • Some tools, like a source code pretty-printer, can relax some of those constraints. While the semantic correctness of the code can be used by the pretty printer, it is not necessary per-se. We can even imagine a pretty-printer working on a syntactically incorrect source, if it ensures not to change its intended meaning.

  • Some other tools, like an IDE, will need to work on code that is incorrect, and even evolving dynamically.

At AdaCore, we already have several interleaved tools to process Ada code: The GNAT compiler, the ASIS library, GNAT2XML, the GPS IDE. A realization of the past years, however, has been that we were lacking a unified solution to process code at the end of the previously described spectrum: Potentially evolving, potentially incorrect Ada code.

Libadalang

Libadalang is meant to fill that gap, providing an easy to use library to syntactically and semantically analyze Ada code. The end-goal is both to use it internally, in our tools and IDEs, to provide the Ada-aware engine, and to propose it to customers and Ada users, so that they can create their own custom Ada aware tools.

Unlike the tools that we currently propose to implement your own tools, Libadalang will provide different levels, allowing a user to work on a purely syntactic level if needed, or access more semantic information.

We will also provide interfaces to multiple languages:

  • You will be able to use Libadalang from Ada of course.

  • But you will also be able to use it from Python, for users who want to prototype easily, or do one off analyses/statistics about their code.

We will in a following series of blog posts showcase how to use Libadalang to solve concrete problems, so that interested people can get a feel of how to use Libadalang.

Stay tuned

Libadalang is not ready for public consumption yet, but you can see the progress being made on GitHub: https://github.com/AdaCore/lib...

Stay tuned!

]]>
New Year's Resolution for 2017: Use SPARK, Say Goodbye to Bugs http://blog.adacore.com/new-years-resolution-for-2017-no-bugs-with-spark Wed, 04 Jan 2017 13:14:00 +0000 Yannick Moy http://blog.adacore.com/new-years-resolution-for-2017-no-bugs-with-spark

NIST has recently published a report called "Dramatically Reducing Software Vulnerabilities" in which they single out five approaches which have the potential for creating software with 100 times fewer vulnerabilities than we do today. One of these approaches is formal methods. In the introduction of the document, the authors explain that they selected the five approaches that meet the following three criteria:

  • Dramatic impact,
  • 3 to 7-year time frame and
  • Technical activities.

The dramatic impact criteria is where they aim at "reducing vulnerabilities by two orders of magnitude". The 3 to 7-year time frame was meant to select "existing techniques that have not reached their full potential for impact". The technical criteria narrowed the selection to the technical area.

Among formal methods, the report highlights strong suits of SPARK, such as "Sound Static Program Analysis" (the raison d'être of SPARK), "Assertions, Pre- and Postconditions, Invariants, Aspects and Contracts" (all of which are available in SPARK), and "Correct-by-Construction". The report also cites SPARK projects Tokeneer and iFACTS as example of mature uses of formal methods.

Another of the five approaches selected by NIST to dramatically reduce software vulnerabilities is what they call "Additive Software Analysis Techniques", where results of analysis techniques are combined. This has been on our radar since 2010 when we first planned an integration between our static analysis tool CodePeer and our formal verification toolset SPARK. We have finally achieved a first step in the integration of the two tools in SPARK 17, by using CodePeer as a first level of proof tool inside SPARK Pro

Paul Black who lead the work on this report was interviewed a few months ago, and he talks specifically about formal methods at 7:30 in the podcast. His host Kevin Greene from US Homeland Security mentions that "There has been a lot of talk especially in the federal community about formal methods." To which Paul Black answers later that "We do have to get a little more serious about formal methods."

NIST is not the only ones to support the use of SPARK. Editor Bill Wong from Electronic Design has included SPARK in his "2016 Gifts for the Techie", saying:

It is always nice to give something that is good for you, so here is my suggestion (and it’s a cheap one): Learn SPARK. Yes, I mean that Ada programming language subset.

For those who'd like to follow NIST or Bill Wong's advice, here is where you should start:

]]>
Make with Ada: DIY instant camera http://blog.adacore.com/make-with-ada-diy-instant-camera Mon, 12 Dec 2016 09:00:00 +0000 Fabien Chouteau http://blog.adacore.com/make-with-ada-diy-instant-camera

There are moments in life where you find yourself with an AdaFruit thermal printer in one hand, and an OpenMV camera in the other. You bought the former years ago, knowing that you would do something cool with it, and you are playing with the latter in the context of a Hackaday Prize project. When that moment comes — and you know it will come — it’s time to make a DIY instant camera. For me it was at the end of a warm Parisian summer day. The idea kept me awake until 5am, putting the pieces together in my head, designing an enclosure that would look like a camera. Here’s the result:


The Hardware

On the hardware side, there’s nothing too fancy. I use a 2 cell LiPo battery from my drone. It powers the thermal printer and a 5V regulator. The regulator powers the OpenMV module and the LCD screen. There’s a push button for the trigger and a slide switch for the mode selection, both are directly plugged in the OpenMV IOs. The thermal printer is connected via UART, while the LCD screen uses SPI. Simple.

The Software

For this project I added support for the OpenMV in the Ada_Drivers_Library. It was the opportunity to do the digital camera interface (DCMI) driver as well as two Omnivision camera sensors, ST7735 LCD driver and the thermal printer.

The thermal printer is only capable of printing black or white pixel bitmap (not even gray scale), this is not great for a picture. Fortunately, the printing head has 3 times more pixels than the height of a QQVGA image, which is the format I get from the OpenMV camera. If I also multiply the width by 3, for each RGB565 pixel from the camera I can have 9 black or white pixels on the paper (from 160x120 to 480x360). This means I can use a dithering algorithm (a very naive one) to produce a better quality image. A pixel of grayscale value X from 0 to 255 will be transformed in a 3x3 black and white matrix, like this:

This is a quick and dirty dithering, one could greatly improve image quality by using the Floyd–Steinberg algorithm or similar (it would require more processing and more memory).

As always, the code is on GitHub.

Have fun!

]]>
Building High-Assurance Software without Breaking the Bank http://blog.adacore.com/formal-methods-webinar-building-high-assurance-software-without-breaking-the-bank Tue, 06 Dec 2016 05:00:00 +0000 AdaCore Admin http://blog.adacore.com/formal-methods-webinar-building-high-assurance-software-without-breaking-the-bank

AdaCore will be hosting a joint webcast next Monday 12th December 2pm ET/11am PT with SPARK experts Yannick Moy and Rod Chapman. Together, they will present the current status of the SPARK solution and explain how it can be successfully adopted in your current software development processes.

Attendees will learn:

  • How to benefit from formal program verification
  • Lessons learned from SPARK projects
  • How to integrate SPARK into existing projects
  • Where to learn about SPARK
  • Why "too hard, too costly, too risky" is a myth

The late computer scientist Edsger Dijkstra once famously said "Program testing can be used to show the presence of bugs, but never to show their absence." This intrinsic drawback has become more acute in recent years, with the need to make software "bullet proof" against increasingly complex requirements and pervasive security attacks. Testing can only go so far. Fortunately, formal program verification offers a practical complement to testing, as it addresses security concerns while keeping the cost of testing at an acceptable level.

Formal verification has a proven track record in industries where reliability is paramount, and among the available technologies, the SPARK toolset features prominently. It has been used successfully for developing high confidence software in industries including Aerospace, Defense, and Air Traffic Management. SPARK tools can address specific requirements for robustness (absence of run-time errors) and functional correctness (contract-based verification) that are relevant in critical systems, including those that are subject to certification requirements.

You can join us for this webinar by registering here.


]]>
Make With Ada Winners Announced! http://blog.adacore.com/make-with-ada-winners-announced Tue, 29 Nov 2016 05:00:00 +0000 AdaCore Admin http://blog.adacore.com/make-with-ada-winners-announced

Judging for the first annual Make with Ada competition has come to an end and we can now reveal the results.

Huge congratulations to Stephane Carrez who came in 1st place this year, gaining a prize of €5000, with his EtherScope project!

The EtherScope monitoring tool analyses Ethernet traffic and can read network packets (TCP, UDP, IGMP, etc.), performs real-time analysis, and displays the results on a 480x272 touch panel. It runs on a STM32F746 micro-controller from STMicroelectronics and its interface can filter results at different levels and report various types of information. 

All sources for the application are available on Github here


In 2nd place, winning €2000, is German Rivera with his Autonomous Car Framework! The framework was developed in Ada 2012 for developing control software for the NXP cup race car. The system is made up of a ‘toy size’ race car chassis, paired with a FRDM-KL25Z board and a TFC-shield board. The car kit also has two DC motors for the rear wheels, a steering servo for the front wheels and a line scan camera.

In 3rd place, winning €1000, is Shawn Nock with his Bluetooth Beacons project! His “iBeacon” project targeted a Nordic Semiconductor nRF51822 System-on-a-Chip (a Cortex-M0 part with integrated 2.4GHz Radio) with a custom development board.    

The two runners up received a Crazyflie 2.0 nano drone each as special prizes. The Ada Lovelace Special Prize was awarded to Sébastien Bardot for inventiveness with his explorer and mapper robot.

The Robert Dewar Special Prize was awarded to German Rivera for dependability with his IoT Networking Stack for the NXP FRDM-K64F board.

Participants had to design an embedded software project for an ARM Cortex M or R processor, using Ada and/or SPARK as the main programming language and put it into action. 

Well done to all the winners and participants, we received some great projects and got the chance to review some interesting ideas this year.

To keep up to date with future competitions and all things Ada, follow @adaprogrammers on Twitter or visit the official Make with Ada website at www.makewithada.org.

]]>
Integrate new tools in GPS (2) http://blog.adacore.com/integrate-new-tools-in-gps-2 Tue, 22 Nov 2016 10:23:00 +0000 Emmanuel Briot http://blog.adacore.com/integrate-new-tools-in-gps-2

Customizing build target switches

In the first post in this series (Integrate new tools in GPS) we saw how to create new build targets in GPS to spawn external tools via a menu or toolbar button, and then display the output of that tool in its own console, as well as show error messages in the Locations view.

The customization left to the user is however limited in this first version. If the build target's Launch Mode was set to Manual With Dialog, a dialog is displayed when the menu or button is selected. This dialog lets users edit the command line just before the target is executed.

The next time that the dialog is open, it will have the modified command line by default.

This is however not very user friendly, since the user has to remember all the valid switches for the tool. For the standard tools, GPS provides nicer dialogs with one widget for each valid switch, so let's do that for our own tool.

Whenever we edit targets interactively in GPS, via the /Build/Settings/Targets menu, GPS saves the modified build target in a local file, which takes precedence over the predefined targets and those defined in plug-ins. This ensure that your modifications will always apply, but has the drawback that any change in the original target is not visible to the user.

So let's remove the local copy that GPS might have made of our build target. The simplest, for now, is to remove the file

  • Windows:  %USER_PROFILE%\.gps\targets.xml
  • Linux and Mac: ~/.gps/targets.xml

Describing the switches

We now have to describe what are the valid switches for our tool.

For a given tool (for instance my_style_checker), we could have multiple build targets. For instance:

  • one that is run manually by the user via a toolbar button
  • and another build target with a slightly different command line that is run every time the user saves a file.

To avoid duplication, GPS introduces the notion of a target model. This is very similar to a build target, but includes information that is shared by all related build targets. We will therefore modify the plugin we wrote in the first part to move some information to the target model, and in addition add a whole section describing the valid switches.

import GPS
GPS.parse_xml("""
    <target-model name="my_style_checker_model">
        <description>Basis for all targets that spawn my_style_checker</description>
        <iconname>gps-custom-build-symbolic</iconname>
        <switches command="my_style_checker" columns="2" lines="1">
           <title column="1" line="1">Styles</title>
           <check label="Extended checks"
                  switch="--extended"
                  tip="Enable more advanced checks"
                  column="1" line="1" />
           <check label="Experimental checks"
                  switch="--experimental"
                  column="1" line="1" />
           <combo switch="--style" separator="=" noswitch="gnat">
              <combo-entry label="GNAT" value="gnat" />
              <combo-entry label="Style 1" value="style1" />
              <combo-entry label="Style 2" value="style2" />
           </combo>

           <title column="2" line="1">Processing</title>
           <spin label="Multiprocessing"
                 switch="-j"
                 min="0"  max="100"  default="1"
                 column="2" line="1" />
        </switches>
    </target-model>

    <target model="my_style_checker_model"
            category="File"
            name="My Style Checker">
        <in-toolbar>TRUE</in-toolbar>
        <in-menu>TRUE</in-menu>
        <launch-mode>MANUALLY_WITH_DIALOG</launch-mode>
        <command-line>
           <arg>my_style_checker</arg>
           <arg>%F</arg>
        </command-line>
        <output-parsers>
           output_chopper
           utf_converter
           location_parser
           console_writer
           end_of_build
        </output-parsers>
    </target>
""")

Let's go over the details.

  • on line 3, we create our target model. To avoid confusion, we give it a name different from that of the build target itself. This name will be visible in the /Build/Settings/Targets dialog, when users interactively create new targets.
  • on line 6, we describe the set of switches that we want to make configurable graphically. The switches will be organized into two columns, each of which contains one group of switches (a "line" in the XML, for historical reasons).
  • on line 7, we add a title at the top of the left column (column "1").
  • on line 8 and 12, we add two check boxes. When they are selected, the corresponding switch is added to the command line. On the screenshot below, you can see that "Extended checks" is enabled, and thus the "--extended" switch has been added. These two switches are  also in column "1".
  • on line 15, we add a combo box, which is a way to let users choose one option among many. When no switch is given on the command line, this is the equivalent of "--switch=gnat". We need to specify that an extra  equal sign needs to be set between the switch and its value.
  • on line 22, we add a spin box, as a way to select a numerical value. No switch on the command line is the equivalent of "-j1".
  • on line 29, we alter the definition of our build target by referencing our newly created target model, instead of the previous "execute", and then remove duplicate information which is already available in the model.

At this point, when the user executes the build target via the toolbar button, we see the following dialog that let's users modify the command line more conveniently.

 The same switches are visible in the /Build/Settings/Targets dialog

Conclusion

In this post, we saw how to further integrate our tool in GPS, by letting users set the command line switches interactively.

We have now reached the limits of what is doable via just build targets. Later posts will extend the plugin with python code, to add support for preferences and custom menus. We will also see how to chain multiple actions via workflows (save all files, compile all code, then run the style checker for instance).

]]>
Integrate new tools in GPS http://blog.adacore.com/integrate-new-tools-in-gps Tue, 15 Nov 2016 15:10:03 +0000 Emmanuel Briot http://blog.adacore.com/integrate-new-tools-in-gps

Integrate external tools in GPS

The GNAT Programming Studio is a very flexible platform. Its goal is to provide a graphical user interface (GUI) on top of existing command line tools. Out of the box, it integrates support for compilers (via makefiles or gprbuild) for various languages (Ada, C, C++, SPARK, Python,...), verification tools (like codepeer), coverage analysis for gcov and gnatcoverage, version control systems (git, subversion,...), models (via QGEN) and more.

But it can't possibly integrate all the tools you are using daily.

This blog describes how such tools can be integrated in GPS, from basic integration to more advanced python based plugins. It is the first in a series that will get your started in writing plug-ins for GPS.

Basic integration: build targets

Every time GPS spawns an external tool, it does this through what is called
a Build Target. They can be edited via the /Build/Settings/Targets menu (see also the online documentation)

Build Targets dialog

On the left of the dialog, you will see a list of all existing build targets, organized into categories. Clicking on any of these shows, in the right panel, the command line used to execute that target, including the name of the tool, the switches,...

The top of the right panel describes how this build targets integrates in GPS. For instance, the Display Target section tells GPS whether to add a  button to the main toolbar, or an entry in the /Build menu or perhaps in some of the contextual menus.

Selecting any of the resulting toolbar buttons or menus will either execute the action directly, no question asked (if the Launch Mode is set to "Manually with no dialog"), or display a dialog to let users modify the switches for that one specific run (if the Launch Mode is set to "Manually with Dialog"). It is even possible to execute that build target every time the user saves a file, which is useful if you want to run some kind of checks.

Creating a new target

To create a new build target, click on the [+] button on the left. This pops up a small dialog asking for the name of the target (as displayed in GPS), a target model (which presets a number of things for the target -- I recommed using "execute -- Run an executable" as a starting point), and finally the category in which the target is displayed.


Create new target

Click on [OK].

In the right panel, you can then edit the command line to run, where the target should be displayed, and how it is run.


Edit target properties

The command line supports a number of macros, like %F, that will be replaced with information from the current context when the target is run. For instance, using "%F" will insert the full path name for the current file. Hover the mouse on the command line field to see all other existing macros.

Press [OK] when you are done modifying all the settings.

(On some versions of GPS, you need to restart GPS before you can execute the build target, or macro expansion will not be done properly)

Target added to toolbar
Target added to menu

Running the target

When you select either the toolbar button or the menu, GPS will open a new console and show the output of your tool. In this example, we have created a simple program that outputs a dummy error on the first line of the file passed in argument.

Running the target

Advanced integration via target models

As you saw in the previous screenshot, the result of running our custom command is that its output is sent to a new window named "Run". But nothing is clickable. In our example, we would like users to be able to click on the location and have GPS display the source location.

This is no doable just via the GUI, so we'll need to write a small plugin. Let's remove the target we created earlier, by once again opening the /Build/Settings/Targets dialog, selecting our new target, and clicking on [-].

Let's then create a new file

  • Windows: %USER_PROFILE%\.gps\plug-ins\custom_target.py
  • Linux and Mac: ~/.gps/plug-ins/custom_target.py

The initial contents of this file is:

import GPS
GPS.parse_xml("""
    <target model="execute" category="File" name="My Style Checker">
        <in-toolbar>TRUE</in-toolbar>
        <in-menu>TRUE</in-menu>
        <launch-mode>MANUALLY</launch-mode>
        <iconname>gps-custom-build-symbolic</iconname>
        <command-line>
            <arg>my_style_checker</arg>
            <arg>%F</arg>
        </command-line>
        <output-parsers>
           output_chopper
           utf_converter
           location_parser
           console_writer
           end_of_build
        </output-parsers>
   </target>
 """)

This file is very similar to what the GUI dialog we used in the first part did (the GUI created a file ~/.gps/targets.xml or %USER_PROFILE%\.gps\targets.xml). See the online documentation.

One major difference though is the list of output parsers on line 12. They tell GPS what should be done with whatever the tool outputs. In our case:

  1. we make sure that the output is not split in the middle of lines (via "output_chopper");
  2. we convert the output to UTF-8 (via "utf_converter");
  3. more importantly for us, we then ask GPS, with "location_parser" to detect error messages and create entries in the Locations view for them, so that users can click them;
  4. we also want to see the full output of the tool in its own console ("console_writer");
  5. Finally, we let GPS performs various cleanups with "end_of_build". Do not forget this last one, since it is also responsible for expanding the %F macro.

The full list of predefined output parsers can be found in the online documentation.

If we then restart GPS and execute our build target, we now get two consoles showing the output of the tool, as happens for compilers for instance.

Locations view

Conclusion

With the little plugin we wrote, our tool is now much better integrated in GPS. We have

  • menus to spawn the tool, with arguments that depend on the current context
  • display the tool's output in a new console
  • make error messages clickable by the user to show the proper sources

In later posts, we will show how to make this more configurable, via custom GPS menus and preferences. We will also explain what workflows are and how they can be used to chain multiple commands.

]]>
Driving a 3D Lunar Lander Model with ARM and Ada http://blog.adacore.com/3d-lunar-lander-model Thu, 10 Nov 2016 14:00:00 +0000 Pat Rogers http://blog.adacore.com/3d-lunar-lander-model

One of the interesting aspects of developing software for a bare-board target is that displaying complex application-created information typically requires more than the target board can handle. Although some boards do have amazing graphics capabilities, in some cases you need to have the application on the target interact with applications on the host. This can be due to the existence of special applications that run only (or already) on the host, in particular.

For example, I recently created an Ada driver for a 9-DOF inertial measurement unit (IMU) purchased from AdaFruit. This IMU device takes accelerometer, magnetometer, and gyroscope data inputs and produces fused values indicating the absolute orientation of the sensor in 3D space. It is sensor-fusion on a chip (strictly, a System in Package, or SiP), obviating the need to write your own sensor fusion code. You simply configure the sensor to provide the data in the format desired and then read the Euler angles, quaternions, or vectors at the rate you require.  I plan to use this orientation data in a small robot I am building that uses an STM32 Discovery board for the control system.

The device is the "AdaFruit 9-DOF Absolute Orientation IMU Fusion Breakout - BNO055" that puts the Bosch BO055 sensor chip on its own breakout board, with 3.3V regulator, logic level shifting for the I2C pins, and a Cortex-M0 to do the actual sensor data fusion.

The breakout board containing the BNO055 sensor and Cortex-M0 processor, courtesy AdaFruit
The bottom of the BNO055 breakout board and a quarter coin, for comparison, courtesy AdaFruit

See https://www.adafruit.com/produ... for further product details.

So I have easy access to fused 3D orientation data but I don't know whether those data are correct -- i.e., whether my driver is really working. I could just display the values of the three axes on the target's on-board LCD screen but that is difficult to visualize in general. 

Again, AdaFruit provides a much more satisfying approach. They have a demonstration for their IMU breakout board that uses data from the sensor to drive a 3D object modeled on the host computer. As the breakout board is rotated, the modeled object rotates as well, providing instant (and much more fun) indication of driver correctness.

The current version of the demonstration is described in detail here, using a web-based app to display the model.  I got an earlier version that uses the "Processing" application to display a model, and instead of using their 3D model of a cat I use a model of the Apollo Lunar Excursion Module (LEM). 

The LEM model is available from NASA, but must be converted to the "obj" model data format supported by the Processing app.  

The Processing app is available here for free. Once downloaded and installed on the host, Processing can execute programs -- "sketches" -- that can do interesting things, including displaying 3D models.  The AdaFruit demo provided a sketch for displaying their cat model. I changed the hard-coded file name to specify the LEM model and changed the relative size of the model, but that was about all that was changed.

The 3D LEM model displayed by the Processing app

The sketch gets the orientation data from a serial port, and since the sensor breakout board is connected to an STM32 Discovery board, we need that board to communicate over one of the on-board USART ports. The Ada Drivers Library includes all that is necessary for doing that, so the only issue is how to connect the board's USART port to a serial port on the host. 

For that purpose I use a USB cable specifically designed to appear as a serial port on the host (e.g., a COM port on Windows). These cables are available from many vendors, including Mouser:

  • Mouser Part No:  895-TTL-232R-5V
  • Manufacturer Part No:  TTL-232R-5V
  • Manufacturer:   FTDI

but note that the above is a 5-volt cable in case that is an issue.  There are 3V versions available.

The end of the cable is a female header, described in the datasheet (DS_TTL-232R_CABLES-217672.pdf).   Header pin 4 on the cable is TXD, the transmit data output. Header pin 5 on the cable is RXD, the receive data input.  The on-board software I wrote that sends the sensor data over the port uses specific GPIO pins for the serial connection, thus I connected the cable header pins to the STMicro board's GPIO pins as follows:

  • header pin 1, the black wire's header slot, to a ground pin on the board
  • header pin 4, the orange wire's header slot, to PB7
  • header pin 5, the yellow wire's header slot, to PB6

On the host, just plug in the USB-to-serial cable. Once the cable is connected it will appear like a host serial port and can be selected within the Processing app displaying the model.  Apply power to the board and the app will start sending the orientation data. 

The breakout board, STM32F429 Discovery board, and USB serial cable connections are shown in the following image. (Note that I connected a green wire to the USB cable's orange header wire because I didn't have an orange connector wire available.)

When we rotate the breakout board the LEM model will rotate accordingly, as shown in the following video:

To display the LEM model, double-click on the "lander.pde" file to invoke the Processing app on that sketch file. Then press the Run button in the Processing app window.  That will bring up another window showing the LEM model. 

In the second window showing the lander, there is a pull-down at the upper left for the serial port selection.  Select the port corresponding to the USB cable attached to the STM32 board's serial port. That selection will be recorded in a file named "serialconfig.txt" located in the same directory as the model so that you don't have to select it again, unless it changes for some reason.

Note that in that second window there is a check-box labeled "Print serial data."  If you enable that option you will see the IMU data coming from the breakout board via the serial port, displayed in the main Processing app window. That data includes the current IMU calibration states so that you can calibrate the IMU (by moving the IMU board slowly along three axes). When all the calibration values are "3" the IMU is fully calibrated, but you don't need to wait for that -- you can start rotating the IMU board as soon as the model window appears.

The Ada Drivers Library is available here.

The Ada source code, the 3D LEM model, and the Processing sketch file for this example are available here.

]]>
Simplifying our product versioning http://blog.adacore.com/simplifying-our-product-versioning Fri, 04 Nov 2016 09:35:00 +0000 Olivier Ramonat http://blog.adacore.com/simplifying-our-product-versioning

Looking at the list of product versions that were expected for 2017 it became clear that we had to review the way we were handling product versioning. Was it so useful to have all these different versions: 1.9.0, 4.9.0, 2.11.0, 2.4.0, 7.5.0, 3.9.0, 17.0.0, 6.3.0, 1.7.0, 3.4.0, 1.4.0, 1.5.0, 3.2.0, 1.2.0, 1.9, 2.13.0, and 2.2.0? Was it worth the cost? Did we really know which product had its version bumped to 3.9.0?

AdaCore products are released on a time based cycle, with the preview release in October, a major release in February, and a corrective version in July. One sensible solution is to choose a date versioning scheme using the year for the major version number, then the minor version for our release increment, and no micro version.

Starting with the 17.0 preview release all AdaCore products will have a unified version number.

By the way, can you guess what products all these version numbers above refer to?

]]>
GNAT On macOS Sierra http://blog.adacore.com/gnat-on-macos-sierra Tue, 11 Oct 2016 11:00:00 +0000 Emmanuel Briot http://blog.adacore.com/gnat-on-macos-sierra

Running GNAT On macOS Sierra

It is this time of the year again. Apple has released a new version of their operating system, now named macOS Sierra.

We started running some tests on that platform, and although we do not have full results yet, things are looking good.

The compiler and most tools work as expected.

However, Apple has once again further restricted what tools can do on the system (for sure that should result in a safer system). The major impact is on our command line debugger, gdb, since this is a tool whose whole purpose is to view and modify running processes on the system. Not something that macOS likes very much in general, although Apple's own debugger of course works flawlessly.

In previous versions of OSX, it was enough for us to codesign gdb. This no longer works, and so far there doesn't seem to be something that we can do on our side (other tools in the Apple ecosystems have similar unresolved issues).

The solutions differ on Sierra 10.12.0 and Sierra 10.12.1.

On Sierra 10.12.0

The solution will require you to slightly lower the security of your system by partially disabling SIP (System Integrity Protection). This can be done as follows:

  1. Reboot your system
  2. Keep command+R pressed until the Apple logo appears on the screen.
  3. Select the menu Utilities/Terminal
  4. Type "csrutil enable --without debug" in the terminal
  5. Finally, reboot your machine again

Note that disabling this will lower the security of your system, so doing the above should really be your decision.

Another impact of this change is that the DYLD_LIBRARY_PATH variable is no longer reset when spawning new processes via the shell. This variable is used by the dynamic linker to find dynamic libraries. It takes precedence over the search path coded in the executables, so is considered as unsafe by the OS. As a result, macOS by default unsets the variable so that the executable you spawn uses its own libraries. We recommend using the DYLD_FALLBACK_LIBRARY_PATH instead, which comes after the application's library search path, in case some libraries are still not found.

On Sierra 10.12.1

The solution requires a patched version of GDB, so either a recent wavefront of GNAT Pro (date >= 2016-11-08) or a fresh snapshot from FSF sources (the patch was committed today 2016-11-09). 

In addition to that, you will need to add the line 'set startup-with-shell off' at the start of your GDB session, which can be done once and for all by copying it to your file .gdbinit on your $HOME. The benefit of putting it in .gdbinit is that it will work with IDEs that launch GDB for you on the program (like GPS, GNATbench or Emacs).


For reference, see Apple's forum.

We will edit this post as we discover more things about macOS Sierra

Edit: The solution for Sierra 10.12.1 still requires user action to avoiding spawning a shell from GDB. We haven't found a better solution yet, we will update the post when there is one.

]]>
Verified, Trustworthy Code with SPARK and Frama-C http://blog.adacore.com/verified-trustworthy-code-with-spark-and-frama-c Wed, 05 Oct 2016 20:06:00 +0000 Yannick Moy http://blog.adacore.com/verified-trustworthy-code-with-spark-and-frama-c

Last week, a few of us at AdaCore attended a one-day workshop organized at Thales Research and Technologies, around the topic of "Verified, trustworthy code - formal verification of software". Attendees from many different branches of Thales (avionics, railway, security, networks) were given an overview of the state-of-the-practice in formal verification of software, focused on two technologies: the SPARK technology that we have been developing at AdaCore, in conjunction with our partner Altran, for programs in Ada; and the Frama-C technology developed at CEA research labs for programs in C.

The two technologies are quite close in terms of methodology and scientific background. My colleague Johannes Kanig recently published a document comparing SPARK with MISRA C and Frama-C, which gives a very good overview of both the similarities and the differences. The Frama-C open source technology is also at the core of the analysis technology developed by TrustInSoft.

The most interesting part of the day was the feedback given by three operational teams who have experimented for a few months with either SPARK (two teams) or Frama-C (one team). The lessons learned by first-time adopters of such technologies are quite valuable:

  1. The fact that the specification language is the same as the programming language (SPARK) or very close to it (Frama-C) makes it easy for developers to write specifications in a few days.
  2. The first application of such technologies in a team should be targeting a not-too-hard objective (for example absence of run-time errors rather than full functional correctness) on a small part of a codebase (for example a few units/files comprosing a few thousand lines of code rather than a complete codebase with hundreds of thousands of lines of code).
  3. Expertise is key to getting started. In particular, there should be an easy way to interact with the tool development/support team. This is all the more important for the first uses, before a process and internal expertise are in place.
  4. While specifying properties is easy, proving them automatically may be hard. It is important here to be able to get feedback from the tool (like counterexamples) when the proof fails, and to have a process in place to investigate proof failures.
  5. Last but not least, a detailed process needs to be described, so that future applications of formal verification (in the same team, or other teams in the same organization) can reliably get the expected results within the expected budget (effort, schedule).


The last point is particularly important to justify the use of new technology in a project. As part of the two experiments with SPARK at Thales, we defined guidelines for the adoption of SPARK in Ada projects, which follow a scale of four levels:

  1. Stone level - valid SPARK - The goal of reaching this level is to identify as much code as possible as belonging to the SPARK subset. The stricter SPARK rules are enforced on a possibly large part of the program, which leads to better quality and maintainability.

  2. Bronze level - initialization and correct data flow - The goal of reaching this level is to make sure no uninitialized data can ever be read and, optionally, to prevent unintended access to global variables. The SPARK code is guaranteed to be free from a number of defects: no reads of uninitialized variables, no possible interference between parameters and global variables, no unintended access to global variables.

  3. Silver level - absence of run-time errors (AoRTE) - The goal of this level is to ensure that the program does not raise an exception at run time. This ensures in particular that the control flow of the program cannot be circumvented by exploiting a buffer overflow, possibly as a consequence of an integer overflow. This also ensures that the program cannot crash or behave erratically when compiled without support for run-time exceptions (compiler switch -gnatp), after an operation that would have triggered a run-time exception. 

  4. Gold level - proof of key integrity properties - The goal of the gold level is to ensure key integrity properties such as maintaining critical data invariants throughout execution, and ensuring that transitions between states follow a specified safety automaton. Together with the silver level, these goals ensure program integrity, that is, the program keeps running within safe boundaries: the control flow of the program is correctly programmed and cannot be circumvented through run-time errors, and data cannot be corrupted.

During the two experiments, we managed to reach bronze, silver and gold levels on various parts of the projects, which allowed both detecting some errors and proving properties about program behavior. A really nice outcome for a few weeks of work.

]]>
Debugger improvements in GPS 17 http://blog.adacore.com/debugger-improvements-in-gps-17 Wed, 05 Oct 2016 09:04:12 +0000 Emmanuel Briot http://blog.adacore.com/debugger-improvements-in-gps-17

The GNAT Programming Studio started as a tool named GVD (graphical visual debugger), which was just a GUI for the gdb command line debugger. Editors were showing the code, but not allowing editing.

Of course, it then evolved (a lot) and became a full fledged integrated environment. But support for debugging activities is still a very important part of the experience. We had accumulated a number of enhancement requests, which have finally been implemented in this year's release.

Here is a quick summary.

Set breakpoints at any time

Historically, users had to initialize the debugger (i.e. enter a special mode in GPS) before they could start setting breakpoints. That's because breakpoints were only managed by the debugger itself. 

In practice though, it is generally when you are reading code that you decide that a breakpoint might be a good idea. Having to start the debugger first breaks the flow of ideas.

So it is now possible to set a breakpoint at any time in GPS, whether the debugger is running or not. To do this, simply click on the line number. GPS will display it with a blue background (and show a tooltip if you hover to explain what the color means). In the following example, we have clicked on line 42.

In previous versions, GPS was displaying small grey or red dots next to lines with executable code, and you had to click on those points to set the breakpoints. We have now removed the display of these dots for several reasons: they were computed via relatively expensive calls to the debugger (and thus required that a debugger be running), they were imprecise, depending on the optimization level used to compile the code, and were taking valuable screen-estate. 

New debugger perspective

Let's now start the debugger. This is done, as before, by selecting the menu /Debug/Initialize/ and the executable to run. This step is still necessary to let GPS know that the debugger should be started, and that you might want to do additional setup before actually running the debugger (like connecting to a board, attaching to a running process,...) We are working on simplifying that step as well, although this will not be part of the release this year.


If you look at the screenshot, you might notice that the layout of the window (what GPS calls the Perspective) is different. Although, if you have used GPS before, you will not see the same view, since GPS always tries to restore what you had setup previously. The solution, currently, is to remove the file perspectives6.xml in HOME/.gps/ or %USER_PROFILE%\.gps depending on your platform and then restart GPS.

Here are the changes we have done:

  • Removed the Data window, which was used to display the variables as boxes in a graphical browser. This view is still available, but has been replaced in the default perspective by the Debugger Variables view (see below)
  • Display the Breakpoints view by default

These are small changes, but might bring attention to some of the view we think are very useful when debugging.

The Debugger Variables view

A new view has been added to display the value of variables as a tree. It comes in addition to the existing Debugger Data. Here are two screenshots showing the same information in both views.

To display information in the Debugger Variables, the simplest is to right-click on the variable in the source editor, then select the Debug/Tree Display contextual menu. This will add the variable to the list of those displayed. You can then expand fields as you want by clicking on the arrows.

When the variable is an access (or a pointer), expand its value will show the value it points to.

If you look at the buttons in the toolbar of the Debugger Variables, you will find some that enable you to display the value of any expression (the + button), or the list of all local variables for the current subprogram, or the value of the CPU registers.

Breakpoints view

We have redone the Breakpoints view, which displays the list of breakpoints. It used to be a very large window but we have now simplified it significantly so that it can be kept visible at all times. And since breakpoints can indeed be set anytime, as we mentioned before, you can also display the  Breakpoint view when the debugger has not been started.

The view lists each known breakpoint, with its location. You can simply double-click on any of the breakpoints to open a source editor showing its location.

By unselecting the check box, the breakpoint will be disabled, and gdb will no longer stop at that location.

If you press the mouse for a short while on the breakpoint (or alternatively click on the gears icon in the toolbar), GPS opens a new dialog that lets you edit some advanced properties of the breakpoint, like the conditions that should be met for the debugger to stop, or how many times the debugger should ignore this breakpoint before it actually stops.

Performance

In the default debug perspective, GPS always displays the Call Stack window which shows, whenever the debugger stops, the current subprogram and where it was called from. This information is of course queried from gdb itself, and depending on the settings it might take a while to compute. For instance, displaying the value of the parameters can sometimes take a few seconds for large callstacks on some systems. This leads to very long delays whenever the debugger stops.

We have optimized things a bit here. If the Call Stack is not configured to show the value of the parameters, then GPS makes sure that gdb doesn't spend any time computing it.

Scripting

Like all parts of GPS, the debugger can be fully controlled via python scripts. This lets you run your own commands whenever the debugger stops for instance, or automatically display the value of some variables, or anything else you could imagine in your context.

In this release, we have added a number of new python commands to make interfacing simpler. For instance, there is now a value_of function to query the value of a variable (while cleaning up extra information output by gdb), or a breakpoints command to efficiently retrieve the current list of breakpoints (and benefit from GPS's own cache)

]]>
Unity & Ada http://blog.adacore.com/unity-ada Mon, 19 Sep 2016 11:30:00 +0000 Quentin Ochem http://blog.adacore.com/unity-ada


Using Ada technologies to develop video games doesn’t sound like an an obvious choice - although it seems like there could be an argument to be made. The reverse, however, opens some more straightforward perspectives. There’s no reason why a code base developed in Ada couldn’t benefit from a game engine to support the user interface. Think of a flight simulator for example, running a mission computer in Ada.

In the past few years, a number of these have been made available to a larger audience, mostly to serve the so-called indie scene. In this blog post, we’ll concentrate on Unity, which has the advantage of using the Mono environment as scripting language. As there is no flight mission computer at AdaCore, we’ll use a slightly modified version, the SPARK/Ada example already ported to a number of environments. So that’ll be a game in Ada after all!

Setup

You can get the code on AdaCore’s Github repository. It has been developed for Windows, but should be easily ported over to Linux (the directive “for Shared_Library_Prefix use "";” in the GNAT Project file may be all you need to remove - see more information in the Ada project setup). The Ada folder contains what’s necessary to build the native code, the TetrisUI contains the Unity project. You will need both GNAT and Unity for everything to work.

Unity can be downloaded from https://unity3d.com/. During the installation, make sure that you select the correct version - 32 bits or 64 bits - matching your GNAT compilation environment. In particular, to date, GNAT GPL is only provided with a 32 bits code generator on Windows, and 64 bits on Linux.

Once the Ada project is built, make sure to copy the resulting library under the Asset/Plugin directory of the Unity project. This will be described in more details below.

Now, let’s dive into the project.

Exporting Tetris Ada to C#

We’re going to skip the section describing the development of the Tetris code. There’s a full explanation available . Note that interestingly, we’re not only interfacing Ada, but actually SPARK code. This provides a good demonstration of a typical situation with Ada, where the core safety-critical functionalities are developed using a higher safety standard (here the core of the game), which can then be integrated in various environments with fewer safety constraints (here the user interface developed in Unity).

As said in the introduction, Unity runs mono - an open-source version of the .Net platform - and therefore allows you to develop scripts using the C# language. So, exporting our Ada code to Unity will turn out to be interfacing Ada and C#.

Step 1 - Setting up the Build

The first step is to set up a library to be loaded within Unity. We need actually several things here: the library has to be dynamic, we need it to be automatically initialized, and we need to make sure that it doesn’t have any dependency on any other libraries, in particular the GNAT libraries. This is achieved through the following settings in the GNAT Project file:

   for Library_Kind use "dynamic";
   for Library_Standalone use "encapsulated";
   for Shared_Library_Prefix use "";

We specify here the fact that the library is dynamic (for Library_Kind use "dynamic";) and that it should encapsulate all dependencies, in particular the GNAT library (for Library_Standalone use "encapsulated";). That second setting is important, otherwise our library would only work if the GNAT run-time is also provided.

By default, the build system (gprbuild) will add a “lib” prefix to the library name. I’m personally working on Windows and don’t fancy having this prefix which doesn’t really look natural. This behavior is cancelled by the clause (for Shared_Library_Prefix use "";). On Linux, where you’re likely to want this prefix, you may need to remove that clause.

Step 2 - Exporting to C, then C#

There’s no direct way to export from Ada to C#, but as the C# Platform Invoke (PInvoke) services allow us to call native C code as often, we’re going to go through C conventions for the interfacing. As a disclaimer, to anyone that is already shaking to the idea of interfacing virtual machines and native code - fear no more. Using PInvoke is surprisingly easier than using something like the Java Native Interface for the jvm - at least for basic interfacing. In our case, for Ada, it will be nothing more than exporting proper symbols.

In order to precisely control what gets exported, we’re going to wrap all calls to the Tetris package (that contains the game core) into a new package Game_Loop (see https://github.com/AdaCore/UnityAdaTetris/blob/master/Ada/src/game_loop.ads and https://github.com/AdaCore/UnityAdaTetris/blob/master/Ada/src/game_loop.adb). This package will have in particular a subprogram “Cycle” responsible for moving pieces around the board. Its interface will look as follows:

  procedure Cycle
     with Export,
     Convention => C,
     External_Name => "tetris_cycle";

I’m using Ada 2012 aspects here, as opposed to the usual “pragma Export”, but the effect is the same. This is declaring the C call convention, exporting the procedure to a C symbol “tetris_cycle”.

Importing this symbol into C# is painfully easy. All we need to do is create a static function of the right profile, and then associate it to the right DllImport directive. We’ll see how to set the C# project in the next section, but here’s the code that will have to be written in the class:

    [DllImport("tetris")]  
    private static extern void tetris_cycle();

And that’s it! We’ll of course need to make sure that the library is at the correct location to be loaded, but again, that’s for the next section.


One last point of importance - exceptions. The core of the game is written in SPARK and all run-time checks have been proven. So we know that when called properly (ie respecting preconditions) this code cannot raise any exception. However, there’s no such checks at the C# to Ada interface level and it’s quite possible that values passed from C# to Ada are wrong. A good example of this is board coordinates, which are constrained between 1 .. X_Size and 1 .. Y_Size but mapped into regular integers. Nothing prevents values in C# to be wrong, and thus to issue an exception when being passed to Ada. The resulting behavior is quite annoying as C# has no way to catch Ada exceptions - Unity will just crash. As it turns out, it happened to me quite a few times before I realized what was happening.

There are ways to throw a C# exception from native code - as to translate an Ada exception into a C# one. The SWIG interface generator does it for example. However, it’s outside of the scope of this blog, so we’ll just make sure that all calls that are part of the interface have default handlers that provide default behaviors. Let’s look at another example from the interface, the code that provides the value on a cell of the board:

  function Get_Kind (X, Y : Integer) return Cell
     with Export,
     Convention => C,
     External_Name => "tetris_get_kind";

 The implementation will look like:

   function Get_Kind (X, Y : Integer) return Cell is
   begin
      return Cur_Board (Y)(X);
   exception
      when others =>
         return Empty;
   end Get_Kind;

Returning the “Empty” literal is far from ideal. There’s no information passed to Unity that something wrong happened. However, this is enough to keep things going.

The C# code will look like:

    [DllImport("tetris")]
    private static extern byte tetris_get_kind(int x, int y);

Integer is quite logically mapped into int. I have to admit that I used implementation knowledge for the return type. I know that it’s a small enumeration that turns into an 8 bits unsigned int, hence the “byte” type. As it turns out however, this implementation knowledge is available to everyone. The -gnatceg switch from GNAT allows to generate a C interface to Ada. The resulting C header files can be directly used to develop C code or to interface with anything (like C#). And as a matter of fact, we could even have used these in the SWIG tool we mentioned before to automatically generate C# and - hey - use their interface to exception handling mechanism!

Back to our work, the last piece worth mentioning here is handling of data structures. More precisely, regarding a type in the Tetris specification:

   type Piece is record
      S : Shape;
      D : Direction;
      X : PX_Coord;
      Y : PY_Coord;
   end record;

We need to be able to access to this type from C#. Again, using -gnatceg we can see that the compiler generates a structure with two unsigned 8 bits integers and 2 regular integers. This is -non portable- insights on the compiler behavior. To do proper interfacing, it would probably have been better to declare this structure and all the types as being C convention. But we’re taking shortcuts here for the purpose of the demonstration.

Interestingly, this type can directly be mapped to C#. C# has two main composite data structures, the “class” and the “struct”. A notable difference is that “class” is pass-by-reference and “struct” pass-by-copy. So a C# struct is very appropriate here and can be directly mapped to:

    private struct Piece
    {
        public byte shape;
        public byte direction;
        public int x;
        public int y;
    }

So that the Ada call:

   function Get_Cur_Piece return Piece
     with Export,
     Convention => C,
     External_Name => "tetris_get_cur_piece";

Becomes:

    [DllImport("tetris")]
    private static extern Piece tetris_get_cur_piece();

One last piece of information, a number of calls are returning booleans value. Recent GNAT compilers will complain when interfacing Ada Boolean type with C - there isn’t such a type in C. Instead of just disregarding the warning we can shut it down by using our own Boolean type (with proper size clause this time):

     type CSharp_Bool is new Boolean with Size => 8;

Used in various places in the interfacing.

In summary, interfacing Ada and C# comes with a couple of hurdles, but is overall relatively painless. The code and this article take a number of shortcuts and present alternatives to the interfacing. Using this in an industrial context would require a bit more care to make sure that the interfaces are portable and safe. And even better, automatic generation of the wrappers and interface would be  ideal (a bit like GNAT-AJIS for Java). The question of exception handling still needs to be worked out but for a demonstrator, that’s good enough. We now have an Ada library ready to be used from within Unity. Let’s play!

Developing the game UI in Unity

Some high level information on Unity

There’s many things to know about Unity and we’re not even going to begin scratching the surface. For more detail, there’s a massive amount of tutorials and books available. We’re only going to provide a high level view of the concepts we deployed in this demonstrator. In order to run the example, the one thing you will need to do is to build the Ada library (see previous section) and to copy it to the Resources asset folder described below.

The first time a Unity project is open, it shows a game scene. There may be many scenes in a game, but for Tetris we’ll only need one. The objects of that scene are listed in the list on the left. These are the ones which are statically defined, but as we are also going to create a couple of objects dynamically, these objects are typed after GameObject. Unity is using an interesting component based design method, where each GameObject is itself composed of components (shape, renderer, position, scripts, etc). Clicking on the Main Camera object, we’ll get the list of these components on the right panel. In particular a Transform at the top, specifying matrix transformation for this object (position, rotation...), a Camera component for what is specific to the camera (in particular switching between orthographic and perspective projections), and at the end a script called Tetris, which will contain the behavior of the game. We decided to associate this script to the camera component, but as a matter of fact, it won’t have to interact with the camera, we could just as well have placed it anywhere.

At the bottom of the screen, we have the Assets. The components and game objects used to build the scene. Two scripts - Tetris that we discussed already and Cell which we’ll see briefly in a bit. A material called “CubeMat” used to render cubes and the Main scene. There are also two directories, Resources and Plugins. In the Asset folder, developers are mostly free of organizing elements the way they want apart from a couple of special directories, these two in particular.

The Plugins directory in particular is responsible for containing the dynamic libraries to be loaded by Unity. When checking the project out, it should be empty. Just drag and drop tetris.dll (or libtetris.so) here to complete the project.

Resources contains objects that can be dynamically instantiated. Here, Cell is going to be one cell on the screen. Clicking on it, you’ll see that it has a position, a collider, a renderer, a material, and a filter (responsible for the shape - or mesh - of the object). It’s also associated with a script also called Cell, which implements some specific services.

And that’s pretty much it. Clicking on the play button on the top, you should be able to launch the game, move bricks with the arrows or accelerate the fall with the space key. Note that stopping and relaunching the game from within the same Unity instance will not reset the board. This is because the library is actually loaded within the Unity environment itself, not in response to the play button. To reset, we can either re-launch Unity, or extend the implementation with some initialization code launched at game start.


The Tetris behavior

The UI for this Tetris is very basic. It’s a 10 * 38 grid of pre-loaded cubes, that we’re going to make appear and disappear depending on the state of the core game. It’s arguably a bit of a shame to use a tool such as Unity and turn it into such a basic implementation - but it has the benefit of adhering to the core developed in Ada, which can then be ported to much more crude environments. It’s all ready to be extended though!

The core script is pretty much all contained in this “Tetris.cs” script, which you can open either from the asset list or by clicking on the entry in the Main camera. You’ll see a bunch of DllImport clauses here, similar to those we described in the previous section. Next is a call to Awake(). Awake is a special function that unity calls once the object has been created and set up. It’s just recognized by Unity from its name, there’s no overriding mechanism used here (although of course C# has support for it). The first line is getting a handle on a prefab, which is one of these resources we’re going to create dynamically:

   prefabBlock = Resources.Load("Cell") as GameObject;

The complete path to the prefab is “Assets/Resources/Cell” but the common prefix is omitted. Once this handle is obtained, I can then instantiate it through instantiate calls:

   Instantiate(prefabBlock)

Which returns a GameObject. As seen before, this GameObject is associated to a script Cell. In Unity, all these scripts are actually descendent of the type Behavior or MonoBeharior. What’s nice about the component based model of unity is that it’s always possible to get any components of a given object from any other component. In other words, I can store this object through a reference to a GameObject or using its Cell directly.

When only one component of a specific type is available, it can be accessed through the generic GetComponent<type>() call, so:

    cells[x, y] = Instantiate(prefabBlock).GetComponent<Cell>();

actually initializes the array cells at x,y position with the instance of the type Cell from the prefab I just initialized. Next we’re going to deactivate the cell for now, as the grid is empty by default. See that we can reference the GameObject from the Cell directly:

    cells[x, y].gameObject.SetActive(false);

The rest of the code is pretty straightforward. Update() is another of these magic functions, this one is called at every frame. We’re calling tetris_cycle() regularly to update the game, and subsequently activating / deactivating cells depending of the status of the board.

The Cell script has some other interesting elements. The first thing it does (in Awake) is to duplicate the material in order to be able to change its color independently of others. The color is then changed in the Renderer component from SetKind. Last but not least, the Explode function creates an instance of the explosion particle system to give a little something when a line is destroyed.

Going Further

There are various things to play with from there. The game itself is merely a proof of concepts, and many things can be added starting with a button to reset the game, a button to exit it, score, etc. The Ada to C# interfacing techniques can also be greatly improved, ideally through automatic binding generation. And of course, it would be nice to have a more comprehensive piece of Ada code to integrate into a larger scale project. All the pieces are here to get started!


]]>
GNAT Programming Studio (GPS) on GitHub http://blog.adacore.com/gnat-programming-studio-gps-on-github Mon, 12 Sep 2016 08:00:00 +0000 Emmanuel Briot http://blog.adacore.com/gnat-programming-studio-gps-on-github

As we mentioned in an earlier post, AdaCore is busy making the source code for its tools available on GitHub. This is not a simple process. Some of the projects first had to be converted to git (from our historical use of subversion), and we needed to make sure that the tools can indeed be built with what is already available out there.

We started with a few libraries and tools, like GtkAda and gprbuild.

We have now managed to make the GNAT Programming Studio (also known as GPS) available. This is a large project that depends on a large number of libraries.

Getting the sources

To download the sources (in fact, to download the whole history of the 16 year old project), head to the GPS GitHub repository. As always on GitHub, you can explore the code online. But the more convenient approach is to click on the green button at the top of the page:

Cloning the sources

Copy the URL, and in your terminal you can then type:

    git clone https://github.com/AdaCore/gps

to get the sources locally. Take a look at the INSTALL file for build instructions. Beware that you will need to first download and compile a number of other dependencies, like gprbuild, GtkAda and GnatColl. That most likely should not be the first project you try to compile.

What you can contribute

We have of course multiple reasons to make our sources available on a day to day basis. The most important is perhaps to help build communities around those tools. This is a great way to make sure they actually fulfill your expectations, and give you a chance to contribute ideas and code if you can.

In the specific case of GPS, we believe we have a great Integrated Development Environment, which can be used in a large number of settings - whether you are developing a multi-million line project, or getting started with your first embedded software, or programming in other languages like C, C++ or Python. We also try to welcome people coming from other environments by taking a look at what exists and when possible, providing similar features.

Such work can be done in GPS via the use of Python plugins, which have a great flexibility nowadays. Take a look at the GPS documentation and the existing plugins for examples.

Unfortunately, we can't do it all and you might be able to help. We have been talking for years of setting up a repository of useful plugins written by people outside of the main GPS developers. And we believe GitHub is a great step forward in that direction.

How you can contribute

The way this works on GitHub is the following:

  1. Create a GitHub account if you do not have one already
  2. Go to the GitHub page for GPS
  3. Click on the Fork button at the top-right corner. A window will pop up that lets you specify where to work. Select your account. Github clones the repository to your own copy.
  4. You can now clone your new repository to your machine.
  5. Modify the sources or add new files with standard git commands, in particular "git add" and "git commit"
  6. When you are ready, type "git push" to contribute your changes to your own repository on GitHub. 
    To make things simpler in the case of trivial changes, steps 4, 5 and 6 can be done directly from the GitHub interface by editing files in place.
  7. You can now let us, the GPS developers, know that you have made the ultimate, ground breaking and fabulous change (or, if you prefer, that you have done a small minor fix). Click on the New Pull Request button. In the resulting dialog, you can leave most of the settings with their default value (which will basically try to merge your master branch into the official GPS master branch). When you press Create Pull Request, the GPS developers will receive a notification that a change is ready for adding to GPS.
  8. We will then do code reviews and comments on your patch and, I am sure, ultimately integrate it in the official GPS sources. At this stage, you have become a GPS contributor!

See the GitHub documentation for more information.

The goal is that after including your changes in GPS, they become part of GPS forever (both the GPL release available for free to everyone, as well as the Pro releases available to our customers). We will retain your names in the commit messages (after all, you did the work).

Looking forward to your contributions!

]]>
Bookmarks in the GNAT Programming Studio (GPS) http://blog.adacore.com/bookmarks-in-the-gnat-programming-studio-gps Wed, 07 Sep 2016 14:00:00 +0000 Emmanuel Briot http://blog.adacore.com/bookmarks-in-the-gnat-programming-studio-gps

Bookmarks have been in GPS for a very long time (first added in 2005). They help you mark places in your source code that you want to easily go back to, even when the code changes.

The description in this blog only apply to very recent development versions of GPS. For our customers, you can immediately request a wavefront (nightly build of GPS) if you want to try them out. For users of the public versions, they will be part of the next GPL 2017 release , so you will have to wait a bit :-)

Basic usage

Creating a new bookmark

The basic usage of bookmarks is as follows: you open a source editor and navigate to the line of interest. You can then create a new bookmark by either using the menu /Edit/Create Bookmark or by opening the Bookmarks view (/Tools/Views/Bookmarks) and then clicking on the [+] button in the local toolbar. In both cases, the Bookmarks view is opened, a new bookmark is created and selected so that you can immediately change its name.

The default name of bookmark is the name of the enclosing subprogram and the initial location of the bookmark (file:line). But you can start typing a new name, and press Enter to finally create the bookmark.

In practice, this is really just a few clicks (one of the menu and press Enter to use the new name), or even just two key strokes if you have set a keyboard shortcut for the menu, via the Preferences dialog.

At any point in time, you can rename an existing bookmark by either clicking on the button in the local toolbar, or simply with a long press on the bookmark itself.

Note the goto icon on the left of the editor line 1646, which indicates there is a bookmark there, as well as the colored mark in the editor scrollbar that helps navigate in the file.

Even though the default name of the bookmark includes a file location, the major benefit of the bookmarks is that they will remain at the same location as the text is edited. In our example, if we add a new subprogram before Display_Splash_Screen, the bookmark will still point at the line containing the call to Gtk_New, even though that line might now be 1700 for instance.

Of course, GPS is not able to monitor changes that you might do through other editors, so in this case the marks might be altered and stop pointing to the expected location.


Adding more bookmarks

Adding more bookmarks

We can create any number of bookmarks, and these have limited impact on performance. So let's do that and create a few more bookmarks, in various files. As you can see in the scrollbar of the editor, we have two bookmarks set in the file bookmark_views.adb, and we can easily jump to them by clicking on the color mark.

But of course, it is much simpler to double-click inside the Bookmarks view itself, on the bookmark of interest to us.

At this point, we have a rather long unorganized list of bookmarks, and it is becoming harder to find them. This is basically what has been available in GPS since 2005, and that doesn't help keeping things organized. So we have recently done a number of improvements that get us far beyond this basic usage.

Organizing bookmarks into groups

Creating groups

When we create new bookmarks, GPS adds them at the top of the list. We might want to organize them differently, which we can do simply with a drag and drop operation: select the bookmark, keep the mouse pressed, and move it to a better place in the list.

Things become more interesting when you drop a bookmark on top of another one. In this case, GPS creates a group that contains the two bookmarks (and that basically behaves like a folder for files). The group is immediately selected so that you can rename it as you see fit.

In our example, we created two groups, corresponding to two features we are working on.

Groups can be nested to any depth, providing great flexibility. So let's create two nested groups, which we'll name TODO, beneath the two we have created. This is a great way to create a short todo list: one top-level group for the name of the feature, then below one group for the todo list, and a few additional bookmarks to relevant places in the code.


Unattached bookmarks, as TODO items

To create these additional groups, we will select the Source editor group, then click on the Create New Group button in the local toolbar, and type "TODO<enter>". This will automatically add the new group beneath Source editor. Let's do the same for the Bookmarks groups. These two groups are empty for now.

Let's add new entries to them. if we already know where code should be added to implement the new todo item, we can do as before: open the editor, select the line, then click on the [+] button. Most often, though, we don't yet know where the implementation will go.

So we want to create an unattached bookmark. Using the name bookmark here is really an abuse of language, since these have no associated source location. But since they are visible in the Bookmarks view, it is convenient to name them bookmarks.

To create them, let's select one of the TODO groups, then select the Create Unattached Bookmark in the local toolbar, and immediately start typing a brief description of the todo. As you can see in the screenshot, these bookmarks do not have a goto icon, since you cannot double click on them to jump to a source location.

When you delete a group, all bookmarks within are also deleted. So once you are done implementing a feature, simply delete the corresponding group to clean up the bookmarks view.

Adding notes

Adding notes to bookmarks

The short name we gave the bookmark is not enough to list all the great ideas we might have for it. Fortunately, we can now add notes to bookmarks, as a way to store more information.

Let's select the "write a blog post" item, then click on the Edit Note button in the local toolbar. This opens a small dialog with a large text area where we can type anything we want. Press Apply to save the text.

Note how a new tag icon was added next to the bookmark, to indicate it has more information. You can view this information in one of three ways:

  • select the bookmark, and click again on the Edit Note button as before
  • double-click on the tag icon.
  • leave the mouse hover the bookmark line. This will display a tooltip with extra information on the bookmark: its name, its current location and any note it might have. This is useful if you only want to quickly glance at the notes for one or more bookmarks

Add note with drag and drop

Dragging text to add a note

Sometimes, though, you want to associate code with the note (i.e. the bookmark should not only point to a location, but you also want to remember the code that was in that location). The simplest to do this is to select the text in the editor, and then drag and drop the selected text directly onto the bookmark. This will create a note (if needed) or add to the existing note the full selected text.

In the tooltips, we use a non-proportional font, so that the code is properly rendered and alignment preserved.




Filtering bookmarks

Filtering the list of bookmarks

If you start creating a lot of bookmarks, and even if you have properly organized them into groups, it might become difficult to find them later on. So we added a standard filter in the local toolbar, like was done already for a lot of other views. As soon as you start typing text in that filter, only the bookmarks that match (name, location or note) are left visible, and all the others are hidden.






Favorite files

Bookmarks used as favorite files

GPS provides a large number of ways to navigate your code, and in particular to open source files. The most efficient one is likely the omni-search (the search field at the top-right corner).

But some users like to have a short list of favorite files that they go to frequently. The Bookmarks view can be used to implement this.

Simply create a new group (here named Favorite files), and create one new bookmark in this group for each file you are interested in. I like to create the bookmark on line 1, but I always remove the line number indication in the name of the bookmark since the exact line is irrelevant here.



Conclusion

The flexibility of the Bookmarks view has been greatly enhanced recently, providing much needed features such as groups of bookmarks, notes, todo items, enhanced tooltips, drag-and-drop operations, ...

We have described two use cases in this post (bookmarks as todo lists and bookmarks for favorite files), but I am sure there are other ways that bookmarks can be used. I would be curious to hear your own ideas in the comments...

]]>
Introducing the Make With Ada competition! http://blog.adacore.com/introducing-make-with-ada Tue, 21 Jun 2016 13:46:00 +0000 AdaCore Admin http://blog.adacore.com/introducing-make-with-ada

If you’ve been looking for a way to start your next embedded project in Ada or SPARK. Then, look no further than the Make with Ada competition!

The embedded software competition is based on the Ada and SPARK languages and the criteria of: software dependability, openness, collaborativeness, and inventiveness.

Until the 30th of September individuals and teams of all abilities are challenged to create a new Ada language based project and log their progress along the way.

Whether you want to explore a different programming approach or demonstrate expertise with the community, we encourage you to register for the first Make with Ada competition.

Why not take some Make with Ada project inspiration from the the latest Ada ARM Cortex-M/R Drivers now available on our GitHub page? Or check out the recent blogs about the candy dispenser, pen plotter and smartwatch hacks from our Paris HQ.

Simply register to take the challenge and start new projects with Ada today! You'll also be in with a chance of winning a share of the prize fund, which totals in excess of 8000 Euros!

For the latest competition updates on Twitter follow @adaprogrammers and use the hashtag #MakewithAda to share your ideas!

]]>
C library bindings: GCC plugins to the rescue http://blog.adacore.com/bindings-gcc-plugins Mon, 13 Jun 2016 13:58:00 +0000 Pierre-Marie de Rodat http://blog.adacore.com/bindings-gcc-plugins

I recently started working on an Ada binding for the excellent libuv C library. This library provides a convenient API to perform asynchronous I/O under an event loop, which is a popular way to develop server stacks. A central part of this API is its enumeration type for error codes: most functions use it. Hence, one of the first things I had to do was to bind the enumeration type for error codes. Believe it or not: this is harder than it first seems!

The enumeration type itself is defined in uv.h and makes a heavy use of C macros. Besides, the representation of each enumerator can be platform dependent. Because of this, I would like to get the type binding automatically generated. But only the enumeration type: I want to bind the rest manually to expose an Ada API that is as idiomatic as possible.

Don’t we already have a tool for this?

The first option that came to my mind was to use the famous -fdump-ada-spec GCC flag:

subtype uv_errno_t is unsigned;
UV_E2BIG : constant uv_errno_t := -7;
UV_EACCES : constant uv_errno_t := -13;
UV_EADDRINUSE : constant uv_errno_t := -98;
--  [...]

That’s a good start but I wanted something better. First, uv_errno_t in C really is an enumeration: each value is independent, there is no bit-field game, so what I want is a genuine Ada enumerated type. Moreover, the output from -fdump-ada-spec is raw and supposed to be used as a starting base to be refined. For instance I would have to rework the casing of identifiers and remove the UV_ prefix.

Besides, the output contains the type definition… but also bindings for the rest of the API (other types, subprograms, …), so I would have to filter out the rest. Overall, given that this binding will have to be generated for each platform, using -fdump-ada-spec does not look convenient.

Cool kids handle C with Clang…

My second thought was to use Clang. Libclang’s Python API makes it possible to visit the whole parse tree, which sounds good. I would write a script to find the enumeration type definition, then iterate over its children so that I could list the couples of names and representation values.

Unfortunately, this API does not expose key data such as the value of an integer literal. This is a deal breaker for my binding generator: no integer literal means no representation value.

And now, what? Building a dummy compilation unit with debug information and extract enumeration data out of the generated DWARF looks cumbersome. No please… don’t tell me I'll have to parse preprocessed C code using regular expressions! Then I remembered something I heard some time ago, a technology existed called… GCC plugins.

Sure, I could probably have done this with Clang’s C++ API, but people working with Ada are much more likely to have GCC available rather than a Clang plugin development setup. Also, I was looking for an opportunity to check how the GCC plugins worked. ☺

… but graybeards use GCC plugins

Since GCC 4.5, it is possible for everyone to extend the compiler without re-building GCC, for instance using your favorite GNU/Linux distribution’s pre-packaged compiler. The GCC wiki lists several plugins such as the famous DragonEgg: an LLVM backend for GCC. Note that the material here assumes it works with GCC 6 but the plugin events have changed a bit since the introduction of plugins.

So how does that work? Each plugin is a mere shared library (.so file on GNU/Linux) that is expected to expose a couple of specific symbols: a license “manifest” and an entry point function. The most straightforward way to write plugins is to use the same programming language as GCC: C++. This way, access to all internals is going to look like GCC’s own code. Note that plugins exist to expose internals in other languages: see for instance MELT or Python.

Loading a plugin is easy: just add a -fplugin=/path/to/plugin.so argument to your gcc/g++ command line: see the documentation for more details.

Now, remember the compiler pipeline architectures that you learnt at school:

Typical compiler pipeline

The GCC plugin API makes it possible to run code at various points in the pipeline: before and after individual function parsing, after type specifier parsing and before each optimization pass, etc. At plugin initialization, a special function is executed and it registers which function should be executed, and when in the pipeline (see API documentation).

But what can plugins actually do? I would say: a lot… but not anything. Plugins interact directly with GCC’s internals, so they can inspect and mutate intermediate representations whenever they are run. However, they cannot change how existing passes work, so for instance plugins will not be able to hook into debug information generation.

Let’s bind

Getting back to our enumeration business: it happens that all we need to do is to process enumeration type declarations right after they have been parsed. Fantastic, let’s create a new GCC plugin. Create the following Makefile:

# Makefile
HOST_GCC=g++
TARGET_GCC=gcc
PLUGIN_SOURCE_FILES= bind-enums.cc
GCCPLUGINS_DIR:= $(shell $(TARGET_GCC) -print-file-name=plugin)
CXXFLAGS+= -I$(GCCPLUGINS_DIR)/include -fPIC -fno-rtti -O2 -Wall -Wextra

bind-enums.so: $(PLUGIN_SOURCE_FILES)
        $(HOST_GCC) -shared $(CXXFLAGS) $^ -o $@

Then create the following plugin skeleton:

/* bind-enums.cc */
#include <stdio.h>

/* All plugin sources should start including "gcc-plugin.h".  */
#include "gcc-plugin.h"
/* This let us inspect the GENERIC intermediate representation.  */
#include "tree.h"

/* All plugins must export this symbol so that they can be linked with
   GCC license-wise.  */
int plugin_is_GPL_compatible;

/* Most interesting part so far: this is the plugin entry point.  */
int
plugin_init (struct plugin_name_args *plugin_info,
             struct plugin_gcc_version *version)
{
  (void) version;

  /* Give GCC a proper name and version number for this plugin.  */
  const char *plugin_name = plugin_info->base_name;
  struct plugin_info pi = { "0.1", "Enum binder plugin" };
  register_callback (plugin_name, PLUGIN_INFO, NULL, &pi);

  /* Check everything is fine displaying a familiar message.  */
  printf ("Hello, world!\n");

  return 0;
}

Hopefully, the code and its comments are self-explanatory. The next steps are to build this plugin and actually run it:

# To run in your favorite shell
$ make
[…]
$ gcc -c -fplugin=$PWD/bind-enums.so random-source.c
Hello, world!

So far, so good. Now let’s do something useful with this plugin: handle the PLUGIN_FINISH_TYPE event to process enumeration types. There are two things to do:

  • create a function that will get executed everytime something fires at this event;
  • register this function as a callback for this event.
/* To be added before "return" in plugin_init.  */
  register_callback (plugin_name, PLUGIN_FINISH_TYPE,
                     &handle_finish_type, NULL);

/* To be added before plugin_init.  */

/* Given an enumeration type (ENUMERAL_TYPE node) and a name for it
   (IDENTIFIER_NODE), describe its enumerators on the standard
   output.  */
static void
dump_enum_type (tree enum_type, tree enum_name)
{
  printf ("Found enum %s:\n", IDENTIFIER_POINTER (enum_name));

  /* Process all enumerators.  These are encoded as a linked list of
     TREE_LIST nodes starting from TYPE_VALUES and following
     TREE_CHAIN links.  */
  for (tree v = TYPE_VALUES (enum_type);
       v != NULL;
       v = TREE_CHAIN (v))
  {
    /* Get this enumerator's value (TREE_VALUE).  Give up if it's not
       a small integer.  */
    char buffer[128] = "\"<big integer>\"";
    if (tree_fits_shwi_p (TREE_VALUE (v)))
      {
        long l = tree_to_shwi (TREE_VALUE (v));
        snprintf (buffer, 128, "%li", l);
      }

    printf ("  %s = %s\n",
            IDENTIFIER_POINTER (TREE_PURPOSE (v)),
            buffer);
  }
}

/* Thanks to register_callback, GCC will call the following for each
   parsed type specifier, providing the corresponding GENERIC node as
   the "gcc_data" argument.  */
static void
handle_finish_type (void *gcc_data, void *user_data)
{
  (void) user_data;
  tree t = (tree) gcc_data;

  /* Skip everything that is not a named enumeration type.  */
  if (TREE_CODE (t) != ENUMERAL_TYPE
      || TYPE_NAME (t) == NULL)
    return;

  dump_enum_type (t, TYPE_NAME (t));
}

These new functions finally feature GCC internals handling. As you might have guessed, tree is the type name GCC uses to designate a GENERIC node. The set of kinds for nodes is defined in tree.def (TREE_CODE (t) returns the node kind) while tree attribute getters and setters are defined in tree.h. You can find a friendlier and longer introduction to GENERIC in the GCC Internals Manual.

By the way: how do we know what GCC passes as the “gcc_data” argument? Well it’s not documented… or more precisely, it’s documented in the source code!

Rebuild the plugin and then run it on a simple example:

$ make
[…]
$ cat <<EOF > test.h
> enum simple_enum { A, B };
> enum complex_enum { C = 1, D = -3};
> typedef enum { E, H } typedef_enum;
$ gcc -c -fplugin=$PWD/bind-enums.so test.h 
Found enum simple_enum:
  A = 0
  B = 1
Found enum complex_enum:
  C = 1
  D = -3

That’s good! But wait: the input example contains 3 types whereas the plugin mentions only the first two, where’s the mistake?

This is actually expected: the first two enumerations have names (simple_enum and complex_enum) while the last one is anonymous. It’s actually the typedef that wraps it that has a name (typedef_enum). The PLUGIN_FINISH_TYPE event is called on the anonymous enum type, but as it has no name, the guard skips it: see the code above: /* Skip everything that is not a named enumeration type. */.

We need names to produce bindings, so let’s process typedef nodes.

/* To be added before "return" in plugin_init.  */
  register_callback (plugin_name, PLUGIN_FINISH_DECL,
                     &handle_finish_decl, NULL);

/* To be added before plugin_init.  */

/* Like handle_finish_type, but called instead for each parsed
   declaration.  */
static void
handle_finish_decl (void *gcc_data, void *user_data)
{
  (void) user_data;
  tree t = (tree) gcc_data;
  tree type = TREE_TYPE (t);

  /* Skip everything that is not a typedef for an enumeration type.  */
  if (TREE_CODE (t) != TYPE_DECL
      || TREE_CODE (type) != ENUMERAL_TYPE)
    return;

  dump_enum_type (type, DECL_NAME (t));
}

The PLUGIN_FINISH_DECL event is triggered for all parsed declarations: functions, arguments, variables, type definitions and so on. We want to process only typedefs (TYPE_DECL) that wrap (TREE_TYPE) enumeration types, hence the above guard.

Rebuild the plugin and run it once again:

$ make
[…]
$ gcc -c -fplugin=$PWD/bind-enums.so test.h 
Found enum simple_enum:
  A = 0
  B = 1
Found enum complex_enum:
  C = 1
  D = -3
Found enum typedef_enum:
  E = 0
  H = 1

Fine, it looks like we covered all cases. What remains to be done now is to tune the plugin so that its output is easy to parse, for instance using the JSON format. Then to write a simple script that turns this JSON data into the expected Ada spec for our enumeration types, this is easy enough but it goes beyond the scope of what is an already long enough post, don’t you think?

Help! My plugin went mad!

So you have started to write your own plugins but something is wrong: unexpected results, GCC internal error, crash, … How can we investigate such issues?

Of course you can do like hardcore programmers do and sow debug prints all over your code. However, if you are more like me and prefer using a debugger or tools such as Valgrind, there is a way. First edit your Makefile so that it builds with debug flags: replace the occurence of -O2 with -O0 -g3. This stands for: no optimization and debug information including macros. Then rebuild the plugin.

Running GDB over the gcc/g++ command is not the next step as it’s just a driver that will spawn subprocesses which perform the actual compilation. Instead, run your usual gcc/g++ command with additional -v -save-temps flags. This will print a lot, including the command lines for the various subprocesses involved:

$ gcc -c -fplugin=$PWD/bind-enums.so test.h -v -save-temps |& grep fplugin
COLLECT_GCC_OPTIONS='-c' '-fplugin=/tmp/bind-enums/bind-enums.so' '-v' '-save-temps' '-mtune=generic' '-march=x86-64'
 /usr/lib/gcc/x86_64-pc-linux-gnu/6.1.1/cc1 -E -quiet -v -iplugindir=/usr/lib/gcc/x86_64-pc-linux-gnu/6.1.1/plugin test.h -mtune=generic -march=x86-64 -fplugin=/tmp/bind-enums/bind-enums.so -fpch-preprocess -o test.i
COLLECT_GCC_OPTIONS='-c' '-fplugin=/tmp/bind-enums/bind-enums.so' '-v' '-save-temps' '-mtune=generic' '-march=x86-64'
 /usr/lib/gcc/x86_64-pc-linux-gnu/6.1.1/cc1 -fpreprocessed test.i -iplugindir=/usr/lib/gcc/x86_64-pc-linux-gnu/6.1.1/plugin -quiet -dumpbase test.h -mtune=generic -march=x86-64 -auxbase test -version -fplugin=/tmp/bind-enums/bind-enums.so -o test.s --output-pch=test.h.gch

The above output contains four lines: let’s forget about line 1 and line 3; line 2 is the preprocessor invocation (note the -E flag), while line 4 performs the actual C compilation. As it's the latter that actually parses the input as C code, it's the one that triggers the plugin events. And it's the one to run under GDB:

# Run GDB in quiet mode so that we are not flooded in verbose output.
$ gdb -q --args /usr/lib/gcc/x86_64-pc-linux-gnu/6.1.1/cc1 -fpreprocessed test.i -iplugindir=/usr/lib/gcc/x86_64-pc-linux-gnu/6.1.1/plugin -quiet -dumpbase test.h -mtune=generic -march=x86-64 -auxbase test -version -fplugin=/tmp/bind-enums/bind-enums.so -o test.s --output-pch=test.h.gch
Reading symbols from /usr/lib/gcc/x86_64-pc-linux-gnu/6.1.1/cc1...(no debugging symbols found)...done.

# Put a breakpoint in your plugin. The plugin is a dynamically loaded shared
# library, so it's expected that GDB cannot find the plugin yet.
(gdb) b plugin_init
Function "plugin_init" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (plugin_init) pending.

# Run the compiler: thanks to the above breakpoint, we will get in plugin_init.
(gdb) run
Starting program: /usr/lib/gcc/x86_64-pc-linux-gnu/6.1.1/cc1 -fpreprocessed test.i -iplugindir=/usr/lib/gcc/x86_64-pc-linux-gnu/6.1.1/plugin -quiet -dumpbase test.h -mtune=generic -march=x86-64 -auxbase test -version -fplugin=/tmp/bind-enums/bind-enums.so -o test.s --output-pch=test.h.gch

Breakpoint 1, plugin_init (plugin_info=0x1de5d30, version=0x1c7fdc0) at bind-enums.cc:63
63        const char *plugin_name = plugin_info->base_name;
(gdb) # Victory!

Debugging can lead you out of your plugin, into GCC’s own code. If this happens, you will need to build your own GCC to include debug information. This is a complex task for which I know, unfortunately, there is no documentation.

Final words

As this small example (final files attached) demonstrates, GCC plugins can be quite useful. This time, we just hooked into some kind of parse tree but it’s possible to deal with all intermediate representations in the compilation pipeline: go and check out what the plugins listed on GCC’s wiki can do!

Attachments

]]>
Make with Ada: ARM Cortex-M CNC controller http://blog.adacore.com/make-with-ada-arm-cortex-m-cnc-controller Wed, 01 Jun 2016 14:03:42 +0000 Fabien Chouteau http://blog.adacore.com/make-with-ada-arm-cortex-m-cnc-controller


I started this project more than a year ago. It was supposed to be the first Make with Ada project but it became the most challenging from both, the hardware and software side.

CNC and Gcode

CNC stands for Computerized Numerical Control. It’s the automatic control of multi-axes machines, such as lathes, mills, laser cutters or 3D printers.

Most of the CNC machines are operated via the Gcode language. This language provides commands to move the machine tool along the different axes.

Here are some examples of commands:

M17 ; Enable the motors
G28 ; Homing. Go to a known position, usually at the beginning of each axis
G00 X10.0 Y20.0 ; Fast positioning motion. Move to (X, Y) position as fast as possible
G01 X20.0 Y30.0 F50.0 ; Linear motion. Move to (X, Y) position at F speed (mm/sec)
G02/G03 X20.0 Y10.0 J-10.0 I0.0 ; Circular motion. Starting from the current position, move to (X, Y) along the circle of center (Current_Position + (I, J))
M18 ; Disable the motors

The Gcode above will produce this tool motion:

Most of the time, machinists will use a software to generate Gcode from a CAD design file. In my case, I used Inkscape (the vector graphics editor) and a plugin that generates Gcode from the graphics. Here is an example of Gcode which I used with my machine: make_with_ada.gcode.

Hardware

To achieve precise movements, CNC machines use a special kind of electric motors: stepper motors. With the appropriate electronic driver, stepper motors rotate by a small fixed angle every time a step signal is received by the driver. (more info).

The rotation motion of the motor is translated to linear motion with a leadscrew. With the characteristic of both, the motor and leadscrew, we can determine the number of steps required to move one millimeter. This information is then used by the CNC controller to convert motion commands into a precise number of steps to be executed on each motor.


To create my own small CNC machine, I used tiny stepper motors from old DVD drives and floppy disk drives. I mounted them on the enclosure of one of the DVD drives. The motors are driven by low voltage stepper drivers from Pololu and the software is running on an STM32F4 Discovery.

Software

My CNC controller is greatly inspired by the Grlb project. Besides the fact that my controller is running on an STM32F4 (ARM Cortex-M4F) and Grbl is running on an Arduino (AVR 8-bit), the main difference is the use of tasking provided by the Ada language and the Ravenscar run-time.

The embedded application is running in 3 tasks:

  1. Gcode interpreter: This task waits for Gcode commands from the UART port, parses them and translates all the motion commands into absolute linear movements (Motion blocks). The circle interpolations are transformed into a list of linear motions that will approximate the circle.
  2. The planner: This task takes motion blocks as an input and splits each one into multiple segments. A segment is a portion of a motion block with a constant speed. By setting the speed of each segment within a block, the planner can create acceleration and deceleration profiles (as seen in the video).
  3. Stepper: This is a periodic task that will generate step signals to the motors. The frequency of the task will depend on the feed rate required for the motion and the acceleration profile computed by the planner. Higher frequency means more steps per second and therefore higher motion speed.

Gcode simulator and machine interface

To be able to quickly evaluate my Gcode/Stepper algorithm and to be able to control my CNC machine, I developped a native (Linux/Windows) application.

The center part of the application shows the simulated motions from the Gcode.The top left button matrix provides manual control of the CNC machine, for example ‘move left’ by 10 mm. The left text view shows the Gcode to be simulated/executed. The bottom text view (empty on this screenshot) shows the message sent to us by the machine.

The Ada code running in the microcontroller and the code running in the simulation tool are 100% the same. This allows for very easy development, test and validation of the Gcode interpreter, motion planner and stepper algorithm.

Give me the code!!!

As with all the Make with Ada projects, the code is available on GitHub: here. Fork it, build it, use it, improve it.

]]>
Certification and Qualification http://blog.adacore.com/certification-and-qualification Mon, 18 Apr 2016 13:43:00 +0000 AdaCore Admin http://blog.adacore.com/certification-and-qualification

AdaCore provides several tools with certification and qualification capabilities, for the rail and avionics industry. Quentin Ochem’s presentation on “Certification and Qualification” at the 2015 AdaCore Tech Days in Boston, Massachusetts provided more information about these two standards, namely DO-178C and EN:50128:2011.

A recent case study from the UK based industry leader, UTC aerospace systems, shows how they are using CodePeer to facilitate the DO-178B certification of their digital terrain system. Our CodePeer tool was selected as it does not report false negatives and it also reduces the need to manually review the code through its static analysis features. Ultimately, the code reviews were carried out in the most efficient and effective way. CodePeer has also been qualified as a class T2 tool for data and control flow analysis under EN:50128 for rail systems.

Our latest guide to how AdaCore's many tools and technologies can be used in conjunction with EN:50128:2011 CENELEC is now available to view on our website.

If you want to be at this year’s AdaCore Tech Days, and hear about similar topics, you can join us either on the 21st - 22nd September in Boston, Massachusetts or on the 6th October in Paris, France. More information can be found here.

]]>
Efficient use of Simics for testing http://blog.adacore.com/efficient-use-of-simics-for-testing Wed, 13 Apr 2016 10:05:25 +0000 Jérôme Lambourg http://blog.adacore.com/efficient-use-of-simics-for-testing

As seen in the previous blog article, AdaCore relies heavily on virtualisation to perform the testing of its GNAT Pro products for VxWorks.

This involves roughly 350,000 tests that are run each day, including the 60,000 tests that are run on Simics. A typical test involves the following steps:

  1. Compilation of an example, with switches and sources defined in the test;
  2. Start an emulator, transfer the compiled module or RTP on it, potentially with some support files;
  3. Run the module or RTP;
  4. Retrieve the output, potentially with files created on the target;
  5. Exit from the emulator.

Compared to an equivalent test run on a native platform, those steps show two challenges:

  1. Feasibility: we need to add the proper support to enable those steps, in particular the file transfer between the host and the emulated target, and do so reliably.
  2. Efficiency: running the emulator and transferring the files back and forth can lead to a significant overhead, incompatible with the time constraints (24h maximum testing time). Some enhanced techniques are thus needed to speed up those critical steps.

Instrumentation overview

In order to accomplish this testing, we need instrumentation, both on the host side (e.g. on the simulator) and on the guest itself (the VxWorks kernel). The requirements are:

  • Be able to transfer files back and forth between guest and host;
  • Be able to retrieve the output;
  • Automatically execute the test scenario on the target.

File transfer

To support transferring files from/to the host, we used the Simics Builder module of Simics. This allowed us to create a dedicated peripheral with which we can interact to transfer files between the host and a RAM drive on the target.

We chose this solution as this can be very efficient (as opposed to transferring the files using a simulated network) while giving us a very high level of flexibility for the various tests.

The simulated peripheral takes the form of a small set of registers and a buffer. The implementation of such a peripheral as a Simics plugin is pretty straightforward. However, some care must be taken for:

  • The size of the individual registers (32bit or 64bit according to the target system)
  • The endianness of those registers

To copy files back and forth, the device responds to syscalls, with the following functions available:

  • OPEN
  • READ
  • WRITE
  • CLOSE
  • UNLINK
  • LSEEK
  • SHUTDOWN

Any write operation on the first register of the device triggers the syscall.

The 3 other registers contain the arguments, and their expected content depends on the specific syscall.

Below is the implementation of the system call of the simulated device, provided as an example of Simics plugin implementation:

static REG_TYPE do_syscall(hostfs_device_t *hfs)
{
  REG_TYPE  ID   = hfs->regs[SYSCALL_ID].value;
  REG_TYPE  arg1 = hfs->regs[ARG1].value;
  REG_TYPE  arg2 = hfs->regs[ARG2].value;
  REG_TYPE  arg3 = hfs->regs[ARG3].value;
  REG_TYPE  ret  = 0;
  char     *host_buf  = NULL;
  REG_TYPE guest_buf;
  REG_TYPE len;

  switch (ID)
    {
    case SYSCALL_OPEN:
      guest_buf = arg1;
      len = 1024; /* XXX: maximum length of filename string */
      arg2 = open_flags(arg2);

      host_buf = malloc(len);
      /* Convert guest buffer to host buffer */
      copy_from_target(hfs, guest_buf, -1, (uint8_t *)host_buf);
      ret = open(host_buf, arg2, arg3);
      free(host_buf);

      return ret;
      break;

On the VxWorks side, this call is implemented in the kernel:

PHYS_ADDR _to_physical_addr(VIRT_ADDR virtualAddr) {
  PHYS_ADDR physicalAddr;

  vmTranslate(NULL, virtualAddr, &physicalAddr);
  return physicalAddr;
}

#define TO_PHY(addr) _to_physical_addr((VIRT_ADDR)addr)

static uintptr_t
hfs_generic (uintptr_t syscall_id,
             uintptr_t arg1,
             uintptr_t arg2,
             uintptr_t arg3)
{
    uintptr_t *hostfs_register = (uintptr_t *)hostfs_addr();
    if (hostfs_register == 0) return -1;

    hostfs_register[1] = arg1;
    hostfs_register[2] = arg2;
    hostfs_register[3] = arg3;

    /* Write syscall_id to launch syscall */
    hostfs_register[0] = syscall_id;
    return hostfs_register[1];
}

uint32_t hfs_open (const char *pathname, uint32_t flags, uint32_t mode)
{
  VIRT_ADDR tmp = ensure_physical((VIRT_ADDR)pathname, strlen(pathname) + 1);
  VIRT_ADDR buf;

  if (tmp != (VIRT_ADDR)NULL) {
    memcpy ((void*)tmp, pathname, strlen(pathname) + 1);
    buf = tmp;
  } else {
    buf = (VIRT_ADDR)pathname;
  }

  return hfs_generic (HOSTFS_SYSCALL_OPEN, (uintptr_t) TO_PHY(buf), flags,
                        mode);
}

Performance considerations

On a typical server, our target is to run around 6,000 of these tests in less than 45 minutes, which means roughly 2 tests per second.

To achieve this goal, the first thing to do is to maximize the parallelism of the execution of the tests: each test can generally be run independently from one another, and also generally requires a single core. This means that on a server with 16 cores, we should be able to execute 16 tests in parallel. This also means that to achieve the target of 6000 tests in 45 minutes globally, on such server each test should target an execution time of less than 8 seconds to execute (8 * 6000 / 16 = 3000 seconds total execution time for the testsuite, so 50 minutes).

Our first experiments with Simics were pretty far from this target: depending on the simulated platform, it could take between 15 seconds to almost a minute just to start VxWorks. When trying to run several Simics instances in parallel, the numbers went even worse, as a lot of server resources were needed to start the simulation.

From what we could see, this was due to the highly configurable and scriptable nature of Simics, where the full simulated environment is built at startup. Such timings were incompatible with our target total execution time.

To address this issue, the Simics engineers pointed us to a very nice feature: The Simics checkpoint. Basically, it’s a mechanism allowing us to save the state of a Simics simulation and to restore it at will.

The restore is very fast.

So what we do now when we build a VxWorks kernel is to also create a generic Simics checkpoint at the point where the VxWorks kernel has just booted. In the Simics script we use this looks like:

script-branch {
  local $con = $system.console
  $con.wait-for-string "Run_shell"
  stop
  write-configuration "checkpoint" -z
  quit
}

And that’s it. To load the checkpoint in our tests:

read-configuration “checkpoint”

to restore the simulation with VxWorks already booted.

This mechanism pre-elaborates the simulation environment, and drastically reduces both the load on the server and the total startup time.

Conclusion

With this testing environment, we can successfully and efficiently test our compiler for VxWorks. As an example, a complete run of the ACATS test suite, containing ~3700 tests (the ACATS testsuite is the standard test suite for the Ada language) takes roughly 31 minutes on a fast linux server, which meets our target performance of 2 tests per second.

By using Simics, AdaCore can now quickly put in place an efficient quality assurance infrastructure when introducing a new VxWorks target to be supported by its GNAT Pro product, improving time-to-market and the overall quality of its products.

]]>
VectorCAST/Ada: Ada 2012 Language Support http://blog.adacore.com/experience-vectorcast-ada-a-look-at-the-latest-release Thu, 07 Apr 2016 12:17:00 +0000 AdaCore Admin http://blog.adacore.com/experience-vectorcast-ada-a-look-at-the-latest-release

We are pleased to announce that on April 27th our partner, Vector, will host a webinar to showcase their latest VectorCAST/Ada release!

Michael Frank, Vector Software's Ada Architect, will cover the following topics: Stub by Function, Change-Based Testing, Code Coverage Analysis, and Ada Language Support, which incorporates Ada 83, 95, 2005, and 2012.

Check out Vector's event page for more information on the topics and to sign up to the upcoming webinar!

]]>
Provably safe programming at Embedded World http://blog.adacore.com/embedded-world-videos Wed, 16 Mar 2016 14:56:00 +0000 AdaCore Admin http://blog.adacore.com/embedded-world-videos

AdaCore continues to build reliable and secure software for embedded software development tools. Last month, we attended Embedded World 2016, one of the largest conferences of its kind in Europe, to present our embedded solutions and our expertise for safety, and mission critical applications in a variety of domains.

Embedded World 2016, for AdaCore, was centered around four major product releases. Namely, QGen 2.1 our customizable and qualifiable code generator. GNAT Pro 7.4 which incorporated several new embedded targets, amongst other enhancements. CodePeer our deep static analysis tool for Ada. SPARK Pro 16 was also released, which provides enhanced coverage of SPARK 2014 language features and more.

Our experts were on hand to talk through the demos running in Ada and SPARK, that were on display and to give further insight into the new product releases.

Day 2 of the conference saw Embedded Computing Executive VP, Rich Nass, interview AdaCore’s Commercial Team Lead, Jamie Ayre to gain an insight into the board demos on display that were running on iOS, Android, and in a bare board fashion on ARM Cortex M4 & M7.

In addition, the runtimes start from a zero footprint, depending on the customer’s needs and in the past these footprints have been used in applications that have gone into space!



Embedded News.TV, also spoke to Jamie about the game of Tetris built on an Atmel SMART SAM4S MCU, which was recently featured in a list created by ATMEL, naming the top 30 projects for 2015 that gamers will love, and this is how we made it!



The train demo also attracted a lot of attention, and as discussed in the interview the software elements are not completely obvious at first. In fact, the signalling system for the trains was written in the Ada 2012 and SPARK languages. With SPARK able to formally verify the properties of the code, this ensures that the trains will not collide at any point, including when the signal points change.



The self playing 2048 game which runs on a STM32F469N-discovery board on display at Embedded World 2016.


The Candy Dispenser with a Twist also runs on a STM32F469 discovery board with a 800x600 LCD.

]]>
CubeSat continues to orbit the Earth thanks to Ada & SPARK! http://blog.adacore.com/carl-brandon-vid Thu, 10 Mar 2016 14:52:08 +0000 AdaCore Admin http://blog.adacore.com/carl-brandon-vid

Dr Carl Brandon of Vermont Technical College and his team of students used SPARK and Ada to successfully launch a satellite into space in 2013 and it has continued to orbit the Earth ever since! At our AdaCore Tech Days in Boston last year Dr Brandon explained further.

The CubeSat was launched along with several other satellites, built as part of NASA’s 2010 CubeSat Launch Initiative (ELaNa), supporting the educational launch of micro-satellites.

Vermont Technical College’s Satellite was the only one from the same launch to remain in orbit.

But why?

The answer is simple, their satellite was programmed differently to the others as it was built using Ada and SPARK code, as opposed to C. This allowed them to ensure a successful launch and orbit. Ada only has a 10% error rate in comparison to that of C code. The SPARK 2005 toolset (the most advanced at the time of satellite production) has only 1% of the C error rate. More information can be found in a recent article that sets about comparing the two languages.

The team are programming a new satellite, The Lunar Ice Cube, using the updated SPARK 2014 language, which we hope to again see in space in the not too distant future.

Dr Brandon said that his students became competent SPARK users in only a few weeks and you can too with our free AdaCore U SPARK 2014 online courses!



If you want to be at this year’s AdaCore Tech Days, you can join us either on the 21st - 22nd September in Boston, Massachusetts or on the 6th October in Paris, France. More information can be found here.


]]>
Formal Verification of Legacy Code http://blog.adacore.com/formal-verification-of-legacy-code Thu, 10 Mar 2016 14:48:00 +0000 Yannick Moy http://blog.adacore.com/formal-verification-of-legacy-code

Just a few weeks ago, one of our partners reported a strange behavior of the well-known function Ada.Text_IO.Get_Line, which reads a line of text from an input file. When the last line of the file was of a specific length like 499 or 500 or 1000, and not terminated with a newline character, then Get_Line raised an exception End_Error instead of returning the expected string. That was puzzling for a central piece of code known to have worked for the past 10 years! But fair enough, there was indeed a bug in the interaction between subprograms in this code, in boundary cases having to do with the size of an intermediate buffer. My colleague Ed Schonberg who fixed the code of Get_Line had nonetheless the intuition that this particular event, finding such a bug in an otherwise trusted legacy piece of code, deserved a more in depth investigation to ensure no other bugs were hiding. So he challenged the SPARK team at AdaCore in checking the correctness of the patched version. He did well, as in the process we uncovered 3 more bugs.

Identifying the Preconditions

Initially, we extracted the Get_Line function and related subprograms in order to analyze them more easily with our various static analysis tools. Running CodePeer on the code detected some possible range check failure, that upon manual analysis corresponded to code being made unreachable by the recent fix. So we removed that dead code. In order to run SPARK on the code, we started with identifying the preconditions of the various subprograms being called. A critical precondition (leading to raising exception End_Error in the original problem reported by our partner) is that procedure Get_Line is not supposed to be called on an empty file. The same is true for the local procedure Get_Rest used in the function Get_Line. So we added an explicit precondition in both cases:

   procedure Get_Line
     (File : in out File_Type;
      Item : out String;
      Last : out Natural)
   with
     Pre => not End_Of_File (File);

   function Get_Rest (S : String) return String with
     Pre => not End_Of_File (File);

We then ran GNATprove on the code, and it showed places where it could not prove that the precondition was satisfied... which made us discover that the initial fix was incomplete! The same fix was needed in another place in the code to ensure that the precondition above could be satisfied always.

Specifying the Functional Behavior of Get_Line

To go beyond that initial attempt, we had to transform the code to hide low-level address manipulations under SPARK subprogram with appropriate contracts, in order to show that the code calling these subprograms was achieving the desired functionality. For example, the low-level memcpy function from libc was originally imported directly:
   procedure memcpy (s1, s2 : chars; n : size_t);
   pragma Import (C, memcpy);

and called in the code of Get_Line:

   memcpy (Item (Last + 1)'Address,
           Buf (1)'Address, size_t (N - 1));

The code above does not allow writing a suitable contract for memcpy. So we hide the low-level memcpy under a typed version:

   procedure Memcpy
     (S1 : in out String;
      S2 : String;
      From1, From2 : Positive;
      N : Natural)
   is
      pragma SPARK_Mode (Off);
      procedure memcpy (s1, s2 : chars; n : size_t);
      pragma Import (C, memcpy);
   begin
      memcpy (S1 (From1)'Address, S2 (From2)'Address, size_t (N));
   end Memcpy;

The call to memcpy above now passes in the typed variables instead of their addresses:

   Memcpy (S1 => Item, From1 => Last + 1,
           S2 => Buf, From2 => 1, N => N - 1);

This allows us to add a suitable functional contract to the typed version of Memcpy, stating that when N is positive, N characters are copied from string S2 at position From2 to string S1 at position From1:

   procedure Memcpy
     (S1 : in out String;
      S2 : String;
      From1, From2 : Positive;
      N : Natural)
   with
     Pre =>
       (if N /= 0 then
          From1 in S1'Range and then
          From1 + N - 1 in S1'Range and then
          From2 in S2'Range and then
          From2 + N - 1 in S2'Range),
     Contract_Cases =>
       (N = 0  => S1 = S1'Old,
        N /= 0 => (for all Idx in S1'Range =>
                     (if Idx in From1 .. From1 + N - 1 then S1 (Idx) = S2 (Idx - From1 + From2)
                      else S1 (Idx) = S1'Old (Idx))));

Note that we also added a suitable precondition to protect against possible buffer overflows, so that proof with SPARK would allow us to guarantee that no such buffer overflow can ever happen. The above contract is a simple one, that could be expressed directly in terms of the subprogram parameters. Subprograms that deal with the file system cannot be directly specified in this way. So instead we defined a ghost variable The_File to represent the file being read, and a ghost variable Cur_Position to represent the current position being read in this file:

   The_File     : String (Positive) with Ghost;
   Cur_Position : Positive := 1 with Ghost;

We could then define the functional contract of the low-level functions interacting with the file system in terms of their effect on these ghost variables. For example, here is the contract of procedure Fgetc which reads a character from a file (we used a procedure instead of a function in order to be able to update ghost variable Cur_Position in the call, something not allowed in SPARK functions):

   procedure Fgetc (Stream : File_Descr; Result : out Int) with
     Global => (Proof_In => The_File, In_Out => Cur_Position),
     Post => (if The_File (Cur_Position'Old) = EOF_Ch then
                Cur_Position = Cur_Position'Old and then
                Result = EOF
              elsif The_File (Cur_Position'Old) = ASCII.LF then
                Cur_Position = Cur_Position'Old and then
                Result = Character'Pos (ASCII.LF)
              else
                Cur_Position = Cur_Position'Old + 1 and then
                Result = Character'Pos (The_File (Cur_Position'Old)));

It says that Cur_Position is incremented unless newline or end-of-file is reached, and that the Result integer returned is the special EOF value for end-of-file, or the integer matching the position of the character otherwise. The most involved contract is the one of Fgets, which gets multiple characters at a time. Here is what the documentation for function fgets says:

The fgets() function reads at most one less than the number of characters specified by size from the given stream and stores them in the string str. Reading stops when a newline character is found, at end-of-file or error. The newline, if any, is retained. If any characters are read and there is no error, a `\0' character is appended to end the string.

Upon successful completion, fgets() returns a pointer to the string. If end-of-file occurs before any characters are read, it returns NULL and the buffer contents remain unchanged. If an error occurs, it returns NULL and the buffer contents are indeterminate.

We transformed it into a procedure in SPARK, so that it can update ghost variable Cur_Position. Here is the contract we wrote to express the informal specification above:

   procedure Fgets
     (Strng   : in out String;
      N       : Natural;
      Stream  : File_Descr;
      Success : out Boolean)
   with
     Global => (Proof_In => The_File, In_Out => Cur_Position),
     Post => (if Success then

                --  Success means no error and no empty file

                (Ferror (Stream) = 0 and then Fpeek (Stream)'Old /= EOF) and then

                --  Case 1: no EOF nor newline character found

                --  N-1 characters are copied to Strng. Nul character is appended.
                --  Previous characters in Strng are preserved beyond the Nth character.
                --  Cur_Position advances N-1 characters.

                (if No_Char_In_Slice (ASCII.LF, Cur_Position'Old, Cur_Position'Old + N - 2)
                      and then
                    No_Char_In_Slice (EOF_Ch, Cur_Position'Old, Cur_Position'Old + N - 2)
                 then
                    Cur_Position = Cur_Position'Old + N - 1 and then
                    (for all Idx in 1 .. N - 1 =>
                       Strng (Idx) = The_File (Cur_Position'Old + Idx - 1)) and then
                    Strng (N) = ASCII.NUL and then
                    (for all Idx in N + 1 .. Strng'Last =>
                       Strng (Idx) = Strng'Old (Idx))

                 --  Case 2: newline character is found

                 --  Characters up to the newline are copied to Strng. Nul character is
                 --  appended. Previous characters in Strng are preserved beyond the nul
                 --  character. Cur_Position advances to the position of the newline
                 --  character.

                 elsif Has_Char_In_Slice (ASCII.LF, Cur_Position'Old, Cur_Position'Old + N - 2)
                         and then
                       No_Char_In_Slice (EOF_Ch, Cur_Position'Old, Cur_Position'Old + N - 2)
                 then
                    declare LF_Pos = Find_Char_In_Slice (ASCII.LF, Cur_Position'Old, Cur_Position'Old + N - 2) in
                      Cur_Position = LF_Pos and then
                      (for all Idx in Cur_Position'Old .. LF_Pos - 1 =>
                         Strng (Idx - Cur_Position'Old + 1) = The_File (Idx)) and then
                      Strng (LF_Pos - Cur_Position'Old + 1) = ASCII.LF and then
                      Strng (LF_Pos - Cur_Position'Old + 2) = ASCII.NUL and then
                      (for all Idx in LF_Pos - Cur_Position'Old + 3 .. Strng'Last =>
                         Strng (Idx) = Strng'Old (Idx))

                 --  Case 3: EOF is found

                 --  Characters prior to EOF are copied to Strng. Nul character is
                 --  appended. Previous characters in Strng are preserved beyond the nul
                 --  character. Cur_Position advances to the position of EOF.

                 elsif No_Char_In_Slice (ASCII.LF, Cur_Position'Old, Cur_Position'Old + N - 2)
                         and then
                       Has_Char_In_Slice (EOF_Ch, Cur_Position'Old, Cur_Position'Old + N - 2)
                 then
                    declare EOF_Pos = Find_Char_In_Slice (EOF_Ch, Cur_Position'Old, Cur_Position'Old + N - 2) in
                      Cur_Position = EOF_Pos and then
                      (for all Idx in Cur_Position'Old .. EOF_Pos - 1 =>
                         Strng (Idx - Cur_Position'Old + 1) = The_File (Idx)) and then
                      Strng (EOF_Pos - Cur_Position'Old + 1) = ASCII.NUL and then
                      (for all Idx in EOF_Pos - Cur_Position'Old + 2 .. Strng'Last =>
                         Strng (Idx) = Strng'Old (Idx)) and then
                      --  redundant proposition to help automatic provers
                      No_Char_In_String (Strng, ASCII.LF, EOF_Pos - Cur_Position'Old + 1)

                 --  Case 4: both newline and EOF appear

                 --  In our model, we choose that this cannot occur. So we consider only
                 --  cases where EOF or newline character are repeated after the first
                 --  occurrence in the file.

                 else False)

                --  Failure corresponds to those cases where low-level fgets
                --  returns a NULL pointer: an error was issued, or the file is
                --  empty. In the last case Cur_Position is not modified.

                else
                  (Ferror (Stream) /= 0 or else
                  (Fpeek (Stream)'Old = EOF and then Cur_Position = Cur_Position'Old)));

Wow! Not so bad for a simple low-level function everyone uses without paying so much attention. But the contract is actually quite straighforward, if you care to look at it in details. If you do so, you'll realize I've displayed it here using a declare-var-in-expr construct that is not yet available in Ada, but will hopefully be in the next version of the language. It comes handy to make the contract readable here, so I'm using it just for this blog post (not in the actual code).


What's left? Well, we should write the contract we'd like to verify for Get_Line:

   procedure Get_Line
     (File : in out File_Type;
      Item : out String;
      Last : out Natural)
   with
     Global => (Input => (EOF, EOF_Ch, The_File), In_Out => (Cur_Position)),

     --  It is an error to call Get_Line on an empty file

     Pre  => not End_Of_File (File),

     Contract_Cases =>

       --  If Item is empty, return Item'First - 1 in Last

       (Item'Last < Item'First =>
          Last = Item'First - 1,

        --  Otherwise, return in Last the length of the string copied, which
        --  may be filling Item, or up to EOF or newline character.

        others =>
          (Last = Item'First - 1 or Last in Item'Range) and then
          (for all Idx in Item'First .. Last =>
             Item (Idx) = The_File (Idx - Item'First + Cur_Position'Old)) and then
          Cur_Position = Cur_Position'Old + Last - Item'First + 1 and then
          (Last = Item'Last or else
           The_File (Cur_Position) = EOF_Ch or else
           The_File (Cur_Position) = ASCII.LF));

And now, let's run GNATprove! I'm taking a bit of a shortcut here, as there are other intermediate subprograms that need contracts, and loops that require loop invariants, for the proof to be performed automatically by GNATprove. But in the process of interacting with GNATprove to get 100% proof, we discovered two other bugs in the implementation of our trusted Get_Line!

  1. One test "K + 2 > Buf'Last" was incorrect, and needed to be fixed as "K + 2 > N". This caused an incorrect value to be returned for the number of characters read in some cases.
  2. The case where Item is empty was incorrectly handled, so that Last was left uninitialized instead of being set to Item'First prior to returning.

Epilogue

After fixing the two bugs mentioned above, we were able to formally prove with SPARK that Get_Line implemented its contract. Well, a slightly simplified version of Get_Line, where we removed the logic around page marks. As an additional verification, we also wrote an implementation of the low-level functions memcpy, memset, fgets, etc. in terms of their effect on the model of ghost variable The_File and Cur_Position. This allowed us to test the not-so-simple contracts on imported low-level functions, to gain better confidence about their correctness.

Overall, we started with a trusted piece of code that had worked for many years, and we ended up fixing 3 bugs in it. The initial bug was present since the first implementation of the function (not the procedure) Get_Line in 2005. The last two bugs were introduced by a faulty change in 2010, when changing the implementation of Get_Line to use fgets instead of fgetc to speed up the function. Interestingly, the comment for this patch said "No change in behavior and good coverage already, so no test case required." And indeed, it's unlikely that testing would have allowed detecting these bugs. Manual review by compiler experts did not find those bugs either. Only formal verification with SPARK allowed us to pinpoint quickly the places where the logic was subtly flawed.

[Cover image by Rama - Own work, CC BY-SA 2.0 fr, https://commons.wikimedia.org/w/index.php?curid=28...]

]]>
Make with Ada: Candy dispenser, with a twist... http://blog.adacore.com/make-with-ada-candy-dispenser-with-twist Thu, 03 Mar 2016 15:40:00 +0000 Fabien Chouteau http://blog.adacore.com/make-with-ada-candy-dispenser-with-twist

A few months ago, my colleague Rebecca installed a candy dispenser in our kitchen here at AdaCore. I don’t remember how exactly, but I was challenged to make it more… fun.

So my idea is to add a touch screen on which people have to answer questions about Ada or AdaCore’s technology, and to modify the dispenser so that people only get candy when they give the right answer. (Evil, isn’t it?)

For this, I will use the great STM32F469 discovery board with a 800x600 LCD and capacitive touch screen. But before that, I have to hack the candy dispenser....

Hacking the dispenser

The first thing to do is to hack the candy dispenser to be able to control the candy delivery system.

The candy dispenser is made of :

  • A container for the candies
  • A motor that turns a worm gear pushing the candies out of the machine
  • An infrared proximity sensor which detects the user’s hand

My goal is to find the signal that commands the motor and insert my system in the middle of it. When the dispenser will try to turn on the motor, it means a hand is detected and I can decide whether or not I actually want to turn on the motor.

To find the signal controlling the motor, I started by looking at the wire going to the motor and where it lands on the board. It is connected to the center leg of a “big” transistor. Another leg of the transistor is tied to the ground, so the third one must be the signal I am looking for. By following the trace, I see that this signal is connected to an 8 pin IC: it must be the microcontroller driving the machine.

At this point, I have enough info to hack the dispenser, but I want to understand how the detection works and see what the signal is going to look like. So I hook up my logic analyser to each pin of the IC and start recording.

Here are the 3 interesting signals: infrared LED, infrared sensor, and motor control.

The detection works in 3 phases:

  • Wait: The microcontroller is turning on the infra-red LED (signal 01) only 0.2 milliseconds every 100ms.This is to save power as the machine is designed to be battery powered.
  • Detection: When something is detected, the MCU will then turn on the infra-red LED 10 times to confirm that there’s actually something in front of the sensor.
  • Delivery: The motor is turned on for a maximum of 300ms. During that period the MCU checks every 20ms to see if the hand is still in front of the sensor (signal 03).

This confirms that the signal controlling the motor is where I want to insert my project. This way I let the MCU of the dispenser handle the detection and the false positive.

I scratched the solder mask, cut the trace and solder a wire to each side. The green wire is now my detection signal (high when a hand is detected) and the white wire is my motor command.

I ran those two wires with an additional ground wire down to the base of the dispenser and then outside to connect them to the STM32F469 discovery board.

Software

The software side is not that complicated: I use the GPL release of GNAT and the Ravenscar run-time on an ARM Cortex-M on the STM32F469. The green wire coming from the dispenser is connected to a GPIO that will trigger an interrupt, so whenever the software gets an interrupt it means that there’s a hand in front of the sensor. And using another GPIO, I can turn on or off the dispenser motor.

For the GUI, I used one of my toy projects called Giza: it’s a simple graphical tool kit for basic touch screen user interface. The interfaces has two windows. The first window shows the question and there is one button for each of the possible answers. When the player clicks on the wrong answer, another question is displayed after a few seconds; when it’s the right answer, the delivery window is shown. On the delivery window the player can abort by using the “No, thanks” button or put his hand under the sensor and get his candies!

The code is available on GitHub here: https://github.com/Fabien-Chouteau/AMCQ

Now let’s see what happens when I put the candy dispenser back in the kitchen:

]]>
Embedded Product Line Updates http://blog.adacore.com/embedded-product-line-update Wed, 02 Mar 2016 14:53:51 +0000 AdaCore Admin http://blog.adacore.com/embedded-product-line-update

Embedded products are not stand alone, this allows them to have safety, mission critical and real-time requirements that they wouldn’t necessarily have otherwise. The embedded product line provides analyzable, verifiable, and certifiable software for both static and dynamic analysis tools.

Pat Roger’s “Embedded Product Line Update” video is now live on our YouTube channel! So, check out the latest updates about the extent of the language support that is offered.

The video highlights Ravenscar's runtime certifications and pending certifications for the rail, avionic, and space standards.

GNAT Pro 7.4 is the latest in several releases from AdaCore that target this sector.


More information can be found on their product pages GNATPro and GNATPro Safety Critical.

Bare board is already an accessible product that has been made even easier to use, thanks to the collaboration effort for drivers and demos through our Git Hub page.

Moreover, there has been a number of Make with Ada Projects, involving this technology. These, include preventing drone crashes with a Crazyflie 2.0, using the Atmel SAM4S (Cortex M4) to play tetris, STM32F4 Discovery, Lego mindstorms NXT, sweet dispenser ARM boards, Raspberry Pi 2 trains, and smart watch App hacking!



If you want to be at this year’s AdaCore Tech Days, you can join us either on the 21st - 22nd September in Boston, Massachusetts or on the 6th October in Paris, France. More information can be found here.


]]>
QGen 2.1 Release! http://blog.adacore.com/qgen-2-1-release Tue, 23 Feb 2016 08:47:15 +0000 AdaCore Admin http://blog.adacore.com/qgen-2-1-release

Embedded World will see the latest release of QGen, the qualifiable and customisable code generator for Simulink® and Stateflow® models!

QGen is a practical tool that uses a model based system engineering format, designed for use in safety-critical control systems. The model based concept allows the user to go back and alter the pre-existing building blocks instead of having to hand edit code.

The model is supported by MISRA C, SPARK, Simulink® and Stateflow®, with the update providing support for the 2015a/b and .slx formats. QGen can be used inside or outside the Simulink® environment, within which a large subset of blocks are supported.

QGen 2.1 supports several more Simulink® blocks, such as Enumerated Constant and provides an optimized implementation of Switch Blocks. In addition, the update supports commented out/through blocks and MATLAB versions 2008b through to 2015b.

QGen tool qualification material includes: DO-178C, EN 50128, and ISO-26262 TCL3 standards. Also, there is validation against Simulink® simulation.

For a more detailed explanation of QGen, why not check out Tucker Taft’s presentation from the 2015 AdaCore Tech Days?

AdaCore Tech Days 2016

Following the success of the AdaCore Tech Days in 2015. The 2016 AdaCore Tech Days will take place from 21st - 22nd September in Boston Massachusetts and on the 6th October in Paris.

]]>
Formal Verification Made Easy! http://blog.adacore.com/formal-verification-made-easy Wed, 27 Jan 2016 14:38:00 +0000 AdaCore Admin http://blog.adacore.com/formal-verification-made-easy

We are pleased to announce our latest release of SPARK Pro! A product that has been jointly developed alongside our partner Altran and following the global AdaCore Tech Days, you can now see the SPARK 2014 talk, Formal Verification Made Easy by AdaCore’s Hristian Kirtchev, on YouTube.

SPARK 2014 has formed the basis for a number of make with Ada projects, from Formal proof on my wrist, which created Tetris in SPARK on the ARM cortex M4, to a project to rewrite the software of the Crazyflie 2.0 in Ada and SPARK to help prevent drone crashes.

SPARK is based on Ada 2012 and aims to provide software verification using formal methods, in order to prove as much as possible and minimise the amount of testing, or altogether eliminate that need if possible. The high level programming language works especially well in a safety critical context, having achieved the avionics standards DO-178C and DO-333.

The SPARK Pro 16.0 integrated development and verification environment uses a mathematics based analysis technology and provides enhanced coverage of the SPARK 2014 language features. Helping to reduce the certification effort for safety-critical and high-security systems.

Check out the video below to understand what it is, the various applications and how to get started today!

]]>
ERTS and Embedded World conferences 2016 http://blog.adacore.com/erts-embedded-world-2016 Mon, 18 Jan 2016 14:46:19 +0000 AdaCore Admin http://blog.adacore.com/erts-embedded-world-2016

We are pleased to announce that we will be a major sponsor and exhibitor at ERTS, Toulouse and will be exhibiting at Embedded World, Nuremberg in the coming months!

We will be showcasing both our Railway and Crazyflie demos at the conferences, which both use our Ada Development Environment for Bare Board ARM Cortex and are an excellent example of how our software can be used in practice by hobbyists and industry experts, alike. Embedded Development projects such as Make with Ada: From bits to music demonstrate how the Ada language can be used in a completely different context to the typical industry domains and just as effectively.

As gold sponsors at ERTS 2016, you can find us in booth 50 from the 27th to the 29th of January, so pass by and find out more about AdaCore’s latest developments in embedded real time software and verification tools for mission-critical, safety critical, and security critical systems worldwide!

Wednesday 27th January and Thursday 28th January will see a variety of talks from AdaCore on the following topics: Structural Coverage Criteria for Executable Assertions, and Bringing SPARK to C developers.

Ada and SPARK are ideally suited, by design, for the development of critically embedded software and the increasing need for safe and secure software globally.

As exhibitors at Embedded World from the 23rd to the 25th of February, you will have yet another opportunity to see our full product range and to ask our experts how they can help you build safe and secure software that matters!

An estimated 25,000 visitors will attend over the course of the show and we will be amongst 900 exhibitors from around the globe, demonstrating the latest innovations in the embedded sector.

We are looking forward to seeing you in booth 4-149.


]]>
Porting the Ada Runtime to a new ARM board http://blog.adacore.com/porting-the-ada-runtime-to-a-new-arm-board Tue, 12 Jan 2016 15:00:00 +0000 Jérôme Lambourg http://blog.adacore.com/porting-the-ada-runtime-to-a-new-arm-board

As a first article (for me) on this blog, I wanted to show you how to adapt and configure a ravenscar-compliant run-time (full or sfp) to a MCU/board when the specific MCU or board does not come predefined with the GNAT run-time.

To do so, I will use GNAT GPL for ARM ELF and 3 boards of the same family: the STM32F429I-Discovery, the STM32F469I-Discovery, and the STM32F746G-Discovery.

These boards are interesting because:

  • They're fun to use, with lots of components to play with (exact features depends on the board): LCD, touch panel, audio in/out, SD-Card support, Networking, etc.
  • They are pretty cheap.
  • They are from the same manufacturer, so we can expect some reuse in terms of drivers.
  • The first one (STM32F429I-Disco) is already supported by default by the GNAT run-time. We can start from there to add support for the other boards.
  • They differ enough to deserve specific run-time adjustments, while sharing the same architecture (ARMv7) and DSP/FPU (Cortex-M4 & M7)
Photo © STMicroelectronics

So where to start ? First, we need to understand what is MCU-specific, and what is board-specific:

  • Instructions, architecture are MCU specific. GCC is configured to produce code that is compatible with a specific architecture. This also takes into account specific floating point instructions when they are supported by the hardware.
  • Initialization of an MCU is specific to a family (All STM32F4 share the same code, the F7 will need adjustments).
  • The interrupts are MCU-specific, but their number and assignments vary from one minor version to another depending on the features provided by the MCU.
  • Memory mapping is also MCU-specific. However there are differences in the amount of available memory depending on the exact version of the MCU (e.g. this is not a property of the MCU family). This concerns the in-MCU memory (the SRAM), not the potential external SDRAM memory that depends on the board.
  • Most clock configuration can be made board-independant, using the MCU's HSI clock (High Speed Internal clock), however this is in general not desirable, as external clocks are much more reliable. Configuring the board and MCU to use the HSE (High Speed External clock) is thus recommended, but board-specific.

From this list, we can deduce that - if we consider the CPU architecture stable, which is the case here - adapting the run-time to a new board mainly consists in:

  • Adapting the startup code in case of a major MCU version (STM32F7, that is Cortex-M7 based).
  • Checking and defining the memory mapping for the new MCU.
  • Checking and defining the clock configuration for the specific board.
  • Make sure that the hardware interrupts are properly defined and handled.

Preparing the sources

To follow this tutorial, you will need at least one of the boards, the stlink tools to flash the board or load examples in memory, and GNAT GPL for ARM (hosted on Linux or Windows) that can be downloaded from libre.adacore.com.

Install it (in the explanations below, I installed it in $HOME/gnat).

The GNAT run-times for bareboard targets are all user-customizable. In this case, they are located in <install prefix>/arm-eabi/lib/gnat.

The board-specific files are located in the arch and gnarl-arch subfolders of the run-times.

So let's create our new run-time there, and test it. Create a new folder named ravenscar-sfp-stm32f469disco, in there, you will need to copy from the original ravenscar-sfp-stm32f4 folder:

  • arch/
  • gnarl-arch/
  • ada-object-path
  • runtime.xml
  • runtime_build.gpr and ravenscar_build.gpr and apply the following modifications:
$ diff -ub ../ravenscar-sfp-stm32f4/runtime_build.gpr runtime_build.gpr
--- ../ravenscar-sfp-stm32f4/runtime_build.gpr	2016-01-09 14:09:26.936000000 +0100
+++ runtime_build.gpr	2016-01-09 14:10:43.528000000 +0100
@@ -1,5 +1,6 @@
 project Runtime_Build is
   for Languages use ("Ada", "C", "Asm_Cpp");
+  for Target use "arm-eabi";

   for Library_Auto_Init use "False";
   for Library_Name use "gnat";
@@ -8,7 +9,8 @@
   for Library_Dir use "adalib";
   for Object_Dir use "obj";

-  for Source_Dirs use ("arch", "common", "math");
+  for Source_Dirs use
+    ("arch", "../ravenscar-sfp-stm32f4/common", "../ravenscar-sfp-stm32f4/math");

   type Build_Type is ("Production", "Debug");

$ diff -ub ../ravenscar-sfp-stm32f4/ravenscar_build.gpr ravenscar_build.gpr
--- ../ravenscar-sfp-stm32f4/ravenscar_build.gpr	2015-04-30 12:36:37.000000000 +0200
+++ ravenscar_build.gpr	2016-01-09 14:11:37.952000000 +0100
@@ -1,7 +1,9 @@
 with "runtime_build.gpr";

 project Ravenscar_Build is
   for Languages use ("Ada", "C", "Asm_Cpp");
+   for Target use "arm-eabi";

   for Library_Auto_Init use "False";
   for Library_Name use "gnarl";
@@ -10,7 +12,8 @@
   for Library_Dir use "adalib";
   for Object_Dir use "obj";

-  for Source_Dirs use ("gnarl-arch", "gnarl-common");
+   for Source_Dirs use
+     ("gnarl-arch", "../ravenscar-sfp-stm32f4/gnarl-common");

   type Build_Type is ("Production", "Debug");
  • ada_source_path with the following content:
arch
../ravenscar-sfp-stm32f4/common
../ravenscar-sfp-stm32f4/math
../ravenscar-sfp-stm32f4/gnarl-common
gnarl-arch

You are now ready to build your own run-time. To try it out, just do:

$ cd ~/gnat/arm-eabi/lib/gnat/ravenscar-sfp-stm32f469disco
$ export PATH=$HOME/gnat/bin:$PATH
$ gprbuild -p  -f -P ravenscar_build.gpr

If everything goes fine, then a new ravenscar-sfp run-time should have been created.

As it has been created directly within the GNAT default search path, you can use it via its short name (e.g. the directory name) just as a regular run-time: by specifying --RTS=ravenscar-sfp-stm32f469disco in gprbuild's command line for example, or by specifying 'for Runtime ("Ada") use "ravenscar-sfp-stm32f469disco"' in your project file.

$ ls
ada_object_path  adalib  gnarl-arch  ravenscar_build.gpr  runtime_build.gpr
ada_source_path  arch    obj         runtime.xml

Handling the STM32F469I-Discovery:

Let's start with the support of the STM32F469I-Discovery. Being the same MCU major version than the STM32F429, modifications to the run-time are less intrusive than the modifications for the STM32F7,

First, we need to make sure the board is properly handled by gprbuild. For that, we edit runtime.xml and change

type Boards is ("STM32F4-DISCO", "STM32F429-DISCO", "STM32F7-EVAL");
Board : Boards := external ("BOARD", "STM32F4-DISCO");

with:

type Boards is ("STM32F469-DISCO");
Board : Boards := external ("BOARD", "STM32F469-DISCO");

Now we're ready to start the real thing.

Memory mapping and linker scripts

In this step, we're going to tell the linker at what addresses we need to put stuff. This is done by creating a linker script from the base STM32F429-DISCO script:

$ cd arch
$ mv STM32F429-DISCO.ld STM32F469-DISCO.ld
# Additionally, you can cleanup the other STM32*.ld scripts, they are unused by this customized run-time

Next, we need to find the technical documents that describe the MCU. Go to http://st.com and search for "stm32f469NI" (that is the MCU used by the discovery board), and once in the product page, click on "design resources" and check the RM0386 Reference Manual.

From the chapter 2.3.1, we learn that we have a total of 384kB of SRAM, including 64kB of CCM (Core Coupled Memory) at 0x1000 0000 and the remaining at 0x2000 0000.

Additionally, we need to check the flash size. This is MCU micro version specific, and the specific MCU of the STM32F469-Disco board has 2 MB of flash. The STM32 reference manual tells us that this flash is addressed at 0x0800 0000.

So with this information, you can now edit the STM32F469-DISCO-memory-map.ld file:

MEMORY
{
  flash (rx)  : ORIGIN = 0x08000000, LENGTH = 2048K
  sram  (rwx) : ORIGIN = 0x20000000, LENGTH = 320K
  ccm   (rw)  : ORIGIN = 0x10000000, LENGTH = 64K
}

System clocks

The run-time is responsible for initializing the system clock. We need the following information to do this - the various clock settings that are available, and the main clock source.

STMicroelectonics provides a Windows tool to help set up their MCU: STM32CubeMX. Using the tool we can verify the clock settings:

Clock configuration in STM32CubeMX

To properly setup the values, we now need to check the speed of the HSE clock on the board. So back to st.com, search for STM32F469-Disco, and from the product page, download the board's user manual UM1932: Discovery kit with STM32F469NI MCU. From chapter "6.3.1 HSE clock source" check that the HSE clock is running at 8MHz.

Now let's check that the run-time is doing the right thing:

  • arch/setup_pll.adb is responsible for the clock setup
  • gnarl-arch/s-bbpara.ads defines the clock constants
  • arch/s-stm32f.ads define some of the MCU's registers, as well as Device ID constants.

Start by adding the STM32F46x device id in s-stm32f.ads. You can search google for the device id, or use st-util to connect to the board and report the id.

   DEV_ID_STM32F40xxx : constant := 16#413#;
   DEV_ID_STM32F42xxx : constant := 16#419#;
   DEV_ID_STM32F46xxx : constant := 16#434#;
   DEV_ID_STM32F7xxxx : constant := 16#449#;

Now let's check the clock constants in s-bbpara.ads:

   function HSE_Clock
     (Device_ID : STM32F4.Bits_12) return STM32F4.RCC.HSECLK_Range
   is (case Device_ID is
          when STM32F4.DEV_ID_STM32F42xxx => 8_000_000,
          --  STM32F429 Disco board
          when STM32F4.DEV_ID_STM32F7xxxx => 25_000_000,
          --  STM32F7 Evaluation board
          when others => 8_000_000)
          --  STM32F407 Disco board and Unknown device
   with Inline_Always;

We see in s-bbpara.ads that the HSE is OK (we fall in the 'others' case). However the Clock_Frequency constant can be bumped to 180_000_000.

   Clock_Frequency : constant := 180_000_000;
   pragma Assert (Clock_Frequency in STM32F4.RCC.SYSCLK_Range);

Looking now at setup_pll.adb, we can verify that this file does not require specific changes. PLLM is set to 8 to achieve a 1 MHz input clock. PLLP is a constant to 2, so PLLN is evaluated to 360 to achieve the expected clock speed : HSE / PLLM * PLLN / PLLP = 180 MHz.

However, the PWR initialization should be amended to handle the STM32F46 case, and can be simplified as we're creating a run-time specific to the MCU:

$ diff -u ../ravenscar-sfp-stm32f4/arch/setup_pll.adb arch/setup_pll.adb
--- ../ravenscar-sfp-stm32f4/arch/setup_pll.adb	2015-04-30 12:36:37.000000000 +0200
+++ arch/setup_pll.adb	2016-01-09 14:11:11.216000000 +0100
@@ -90,7 +90,6 @@
    procedure Initialize_Clocks is

       HSECLK    : constant Integer := Integer (HSE_Clock (MCU_ID.DEV_ID));
-      MCU_ID_Cp : constant MCU_ID_Register := MCU_ID;

       -------------------------------
       -- Compute Clock Frequencies --
@@ -194,11 +193,7 @@
       --  and table 15 p79). On the stm32f4 discovery board, VDD is 3V.
       --  Voltage supply scaling only

-      if MCU_ID_Cp.DEV_ID = DEV_ID_STM32F40xxx then
-         PWR.CR := PWR_CR_VOS_HIGH_407;
-      elsif MCU_ID_Cp.DEV_ID = DEV_ID_STM32F42xxx then
-         PWR.CR := PWR_CR_VOS_HIGH_429;
-      end if;
+      PWR.CR := PWR_CR_VOS_HIGH_429;

       --  Setup internal clock and wait for HSI stabilisation.
       --  The internal high speed clock is always enabled, because it is the

Interrupts

The available interrupts on the MCU can be found in the Reference Manual.

However, an easier and better way to get the list of interrupts is by generating the Ada bindings from the CMSIS-SVD file for this board using the svd2ada tool that can be found on GitHub, and by downloading the SVD file that corresponds to the current MCU (STM32F46_79x.svd) directly from ARM. This binding generates the interrupts list and we can then check the ones that are not mapped by the current run-time.

$ svd2ada ~/SVD_FILES/STM32F46_79x.svd -p STM32_SVD -o temp
$ cat temp/stm32_svd-interrupts.ads
...
...
   UART7_Interrupt: constant Interrupt_ID := 84;

   UART8_Interrupt: constant Interrupt_ID := 85;

   SPI4_Interrupt: constant Interrupt_ID := 86;

   SPI5_Interrupt: constant Interrupt_ID := 87;

   SPI6_Interrupt: constant Interrupt_ID := 88;

   SAI1_Interrupt: constant Interrupt_ID := 89;

   LCD_TFT_Interrupt: constant Interrupt_ID := 90;

   LCD_TFT_1_Interrupt: constant Interrupt_ID := 91;

   DMA2D_Interrupt: constant Interrupt_ID := 92;

   QUADSPI_Interrupt: constant Interrupt_ID := 93;

A total of 91 interrupts are defined by the MCU, with an additional 2 required by GNAT (Interrupt Id 0 is reserved, and GNAT maps the SysTick interrupt to Id 1).

So let's amend the gnarl-arch/a-intnam.ads file:

   HASH_RNG_Interrupt               : constant Interrupt_ID := 82;
   FPU_Interrupt                    : constant Interrupt_ID := 83; --  This line and below are new
   UART7_Interrupt                  : constant Interrupt_ID := 84;
   UART8_Interrupt                  : constant Interrupt_ID := 85;
   SPI4_Interrupt                   : constant Interrupt_ID := 86;
   SPI5_Interrupt                   : constant Interrupt_ID := 87;
   SPI6_Interrupt                   : constant Interrupt_ID := 88;
   SAI1_Interrupt                   : constant Interrupt_ID := 89;
   LCD_TFT_Interrupt                : constant Interrupt_ID := 90;
   LCD_TFT_1_Interrupt              : constant Interrupt_ID := 91;
   DMA2D_Interrupt                  : constant Interrupt_ID := 92;
   QUADSPI_Interrupt                : constant Interrupt_ID := 93;

end Ada.Interrupts.Names;

We also need to edit arch/handler.S to properly initialize the interrupt vector:

$ diff -bu ../ravenscar-sfp-stm32f4/arch/handler.S arch/handler.S
--- ../ravenscar-sfp-stm32f4/arch/handler.S	2014-09-15 11:28:25.000000000 +0200
+++ arch/handler.S	2016-01-09 11:58:32.456000000 +0100
@@ -145,6 +145,16 @@
 	.word   __gnat_irq_trap      /* 95 IRQ79.  */
 	.word   __gnat_irq_trap      /* 96 IRQ80.  */
 	.word   __gnat_irq_trap      /* 97 IRQ81.  */
+	.word   __gnat_irq_trap      /* 98 IRQ82.  */
+	.word   __gnat_irq_trap      /* 99 IRQ83.  */
+	.word   __gnat_irq_trap      /* 100 IRQ84.  */
+	.word   __gnat_irq_trap      /* 101 IRQ85.  */
+	.word   __gnat_irq_trap      /* 102 IRQ86.  */
+	.word   __gnat_irq_trap      /* 103 IRQ87.  */
+	.word   __gnat_irq_trap      /* 104 IRQ88.  */
+	.word   __gnat_irq_trap      /* 105 IRQ89.  */
+	.word   __gnat_irq_trap      /* 106 IRQ90.  */
+	.word   __gnat_irq_trap      /* 107 IRQ91.  */

 	.text

And we also need to bump the number of interrupt IDs in gnarl-arch/s-bbpara.ads:

   Number_Of_Interrupt_ID : constant := 93;

And that's it

The necessary job has now been done to support the STM32F469I-Disco. You can now install the run-time, and use it with the examples from our bareboard drivers repository on GitHub. Note that, as of the time when this article is written, only the 'svd' branch includes some drivers support for this board.

$ gprbuild -P ravenscar_build.gpr
$ cd ~/bareboard/ARM/STMicro/STM32/examples/balls
$ git checkout svd
$ gprbuild -p -P balls_demo.gpr -XBOARD=STM32F469-DISCO -XRTS=ravenscar-sfp -XLCH=lcd -XLOADER=ROM --RTS=ravenscar-sfp-stm32f469disco
$ arm-eabi-objcopy -O binary obj/demo obj/demo.bin
$ st-flash write obj/demo.bin 0x8000000

Porting the run-time to the STM32F7-DISCOVERY

Now on to the STM32F7. This is going to be a bit more difficult for one reason: the STM32F7, being based on the Cortex-M7, can now benefit from Data and Instruction caches. These caches need explicit initialization. A minimal support for the STM32F7 already exists in the run-time, but it is incomplete as these caches are not properly initialized.

Prepare the run-time

First of all, let's create the new run-time for this board. We'll start this time from the work previously performed for the STM32F469-Discovery board to speed up the process.

$ cd ~/gnat/arm-eabi/lib/gnat
$ cp -r ravenscar-sfp-stm32f469disco ravenscar-sfp-stm32f7disco

Enable Data and Instruction caches

Initialization of the cache is described in details by ARM in the Cortex-M7 processor technical reference manual.

So let's try to update the startup code. For that, we're going to add a new file 'arch/start-common.S':

	.syntax unified
	.cpu cortex-m4
	.thumb

	.text
	.thumb_func
	.globl	_stm32_start_common
        .type _stm32_start_common, #function
_stm32_start_common:
        /**************/
        /* Enable FPU */
        /**************/

        movw     r0,#0xED88
        movt     r0,#0xE000
        ldr      r1,[r0]
        orr      r1,r1,#(0xF << 20)
        str      r1,[r0]

        /* Wait for store to complete and reset pipeline with FPU enabled  */
        dsb
        isb

        /********************
         * Enable I/D cache *
        ********************/

        /* Register definition for cache handling */
        .set    CCSIDR,  0xE000ED80
        .set    CSSELR,  0xE000ED84
        .set    DCISW,   0xE000EF60
        .set    ICIALLU, 0xE000EF50
        .set    CCR,     0xE000ED14

        /* First invalidate the data cache */
dcache_invalidate:
        mov     r1, #0x0
        ldr     r0, =CSSELR
        str     r1, [r0]        /* Select the data cache size */
        dsb

        ldr     r0, =CCSIDR
        ldr     r2, [r0]        /* Cache size identification */
        and     r1, r2, #0x7    /* Number of words in a cache line */
        add     r7, r1, #0x4

        ubfx    r4, r2, #3, #10  /* r4 = number of ways - 1 of data cache */
        ubfx    r2, r2, #13, #15 /* r2 = number of sets - 1 of data cache */
        clz     r6, r4           /* Calculate bit offset for "way" in DCISW */

        ldr     r0, =DCISW

inv_loop1:
        mov     r1, r4
        lsls    r8, r2, r7

inv_loop2:
        lsls    r3, r1, r6
        orrs    r3, r3, r8

        str     r3, [r0]        /* Invalidate the D-Cache line */
        subs    r1, r1, #1
        bge     inv_loop2
        subs    r2, r2, #1
        bge     inv_loop1

        dsb
        isb

        /* Now invalidate the instruction cache */
icache_invalidate:
        mov     r1, #0x0
        ldr     r0, =ICIALLU
        str     r1, [r0]

        dsb
        isb

        /* Finally enable Instruction and Data cache */
        ldr     r0, =CCR
        ldr     r1, [r0]
        orr     r1, r1, #(0x1 << 16) /* Sets the data cache enabled field */
        orr     r1, r1, #(0x1 << 17) /* Sets the i-cache enabled field */
        str     r1, [r0]

        dsb
        isb

        /*****************************
         * TCM Memory initialisation *
        *****************************/

        .set    CM7_ITCMCR, 0xE000EF90
        .set    CM7_DTCMCR, 0xE000EF94

        ldr     r0, =CM7_ITCMCR
        ldr     r1, [r0]
        orr     r1, r1, #0x1 /* set the EN field */
        str     r1, [r0]

        ldr     r0, =CM7_DTCMCR
        ldr     r1, [r0]
        orr     r1, r1, #0x1 /* set the EN field */
        str     r1, [r0]

        dsb
        isb

end:
        bx lr
        .size _stm32_start_common, . - _stm32_start_common

This file initializes the FPU, the data cache, the instruction cache (according to the ARM documentation), as well as the TCM memory.

We now need to call it from the startup files, start-ram.S and start-rom.S.

start-ram.S:

  /* Init stack */
 	ldr	sp,.LinitSp

-        /* Enable FPU */
-        movw     r0,#0xED88
-        movt     r0,#0xE000
-        ldr      r1,[r0]
-        orr      r1,r1,#(0xF << 20)
-        str      r1,[r0]
-
-        /* Wait for store to complete and reset pipeline with FPU enabled  */
-        dsb
-        isb
+        bl _stm32_start_common

 	/* Clear .bss */
 	movw	r0,#:lower16:__bss_start

start-rom.S:

 _start_rom:
-        /* Enable FPU */
-        movw     r0,#0xED88
-        movt     r0,#0xE000
-        ldr      r1,[r0]
-        orr      r1,r1,#(0xF << 20)
-        str      r1,[r0]
+        bl _stm32_start_common

Clocks, interrupts, linker scripts, etc.

We will also create a linker script for the STM32F7, and add the new board to runtime.xml. We perform the same run-time modifications we did for the STM32F469-Disco board:

create arch/STM32F7-DISCO-memory-map.ld:

MEMORY
{
itcm (x) : ORIGIN = 0x00000000, LENGTH = 16K
flash (rx) : ORIGIN = 0x08000000, LENGTH = 1024K
dtcm (rx) : ORIGIN = 0x20000000, LENGTH = 64K
sram (rwx) : ORIGIN = 0x20010000, LENGTH = 240K
}

In s-stm32f.ads, DEV_ID_STM32F7xxxx is already defined.

In s-bbpara.ads, the HSE clock is also properly set to 25MHz, the MCU can run at 216 MHz, but STM32CubeMX shows some issues with such value, so we simplify by using a 200MHz value.

Now edit runtime.xml:

type Boards is ("STM32F7-DISCO");
Board : Boards := external ("BOARD", "STM32F7-DISCO");

The interrupts are very similar between the STM32F746 and the STM32F469, so you can benefit from the changes already performed.

Et voilà. Now you can rebuild the run-time, and test it similarly to the stm32f469-disco.

$ gprbuild -P ravenscar_build.gpr
$ cd ~/bareboard/ARM/STMicro/STM32/examples/balls
$ gprbuild -p -P balls_demo.gpr -XBOARD=STM32F7-DISCO -XRTS=ravenscar-sfp -XLCH=lcd -XLOADER=ROM --RTS=ravenscar-sfp-stm32f7disco
$ arm-eabi-objcopy -O binary obj/demo obj/demo.bin
$ st-flash write obj/demo.bin 0x8000000
GNAT on the three boards

Final words and refinements

You will find below the source files for the runtimes.

Although part of the initial run-time for the STM32F429-Disco is delivered with GNAT, it is not necessarily well optimized (some missing interrupts and a non-optimal clock speed in particular). So I included the sfp and full ravenscar run-times for it as well in the final source packages.

Also, in the attached source package, I made use of extending projects to adjust the runtimes. The setup is a bit complex so I haven't explained it above as this is not really part of the subject, but you can have a look if you want. By using extending projects, the advantage is that I only needed to add the files that I'm actually modifying, and thus can more easily benefit from a futur upgrade of GNAT.

Finally, in the downloadable sources, I got rid of the 'BOARD' scenario variable, as the runtimes are now board specific: such scenario variable is only useful when supporting a complete board family.

To go further in customized run-time, you can refer to the following documentation: Customized run-time.

Attachments

]]>
Ada Lovelace Bicentennial http://blog.adacore.com/ada-lovelace-bicentiennial Fri, 18 Dec 2015 15:26:00 +0000 Yannick Moy http://blog.adacore.com/ada-lovelace-bicentiennial

The three of us attended the Ada Lovelace Symposium in Oxford (UK). The two days were one fantastic discovery after another about the life, achievements and legacy of Ada Lovelace, the programming pioneer who lent her name to the Ada language.

Ada's birthday cake with a design by Sydney Padua

The centrepiece of the event was the famous first scientific article of Ada Lovelace about the workings of the Analytical Engine, the first generic programmable machine invented by Charles Babbage. Artists and scholars from both science and humanities departments provided insightful perspectives on both the scientific, technical and philosophical aspects of Ada Lovelace's most celebrated work, justifying her position as an icon of women in STEM.

Let's speak out our heart about what we saw and did there!

Emma:

As a woman, I came away from the Ada Lovelace symposium feeling truly inspired. Its wonderful to not only celebrate, but to deeply understand Ada’s life and the magnitude of her achievements in STEM fields. She became alive in front of our eyes, thanks to the great breadth of content presented by scholars, intellects, mathematicians and enthusiasts.

Insight into how machines first connected with the world allowed us to bridge the gaps between computer programming then and now, and recognise her for the pioneer of programming that she really was. It is fair to say that the in-depth research into The Analytical Engine left the audience engrossed and captivated by it's complex process yet simple objective. A personal highlight was organiser, Ursula Martin's talk into the Ada archives, with specific reference to her childhood aspirations giving us a glimpse into her life before her collaborations with Babbage.

Ada as a child, portrait on display in Somerville College, Oxford

If you are a graphic novel lover, I strongly recommend reading comic author, Sydney Padua's The Thrilling Adventures of Lovelace and Babbage, a brilliant balance between fact and fantasy with extensive footnotes and geeky, graphic humour. For a fascinating account into it's execution and conception, check out Padua's talk here - Ada Lovelace 5 at 1:03:40.

Here's to the tricentennial!

Jamie:

What a wonderful way to bring together so many people passionate about their individual areas of expertise. I particularly enjoyed the way the conference investigated how the multi-faceted personality of Ada influenced her era and ours. Indeed, I came away feeling that I now know her more intimately. And yet, as with any good intellectual debate, there are still questions left unanswered and I shall be delving into some of the books referenced by the presenters.

Our favourite programming language was well represented in a brief but very witty talk by John Barnes (should you see him, ask him to tell you his anecdote about Byron's bear!). More generally, Ada was mentioned in a large number of the presentations and truly given the respect it deserves.

All in all a fantastic event and I hope it wont be too long before I'm able to attend another one!

Yannick:

All talks were of great depth, on many different aspects of Ada Lovelace's work and life. As talks unfolded, we could see her more and more alive in the weaving of bad and good events in her personal life (in particular she was abandonned by her famous father Lord Byron at the age of one month and never saw him again) with her scientific and technical work culminating in the collaboration with Charles Babbage. Speaking of her scientific achievements, various presenters surveyed her role at the side of Babbage, the actual significance of her famous article, and her scientific proficiency based on her communications with De Morgan. The vision of Ada Lovelace that will remain with me after these very detailed investigations is one of an idealistic and passionate scientific eager to make an impact in the world for the benefit of humanity. That Babagge was really the first programmer in the world, but that she was the second. That their fruitful collaboration is an early example of Research & Development projects at its best. The fact that the Analytical Machine was never completed is secondary, a result of the mismatch between the great vision of Babagge & Lovelace and the technical realities of the their age. Computer pioneers from Turing to Aiken explicitly recognized their inheritage from these early pioneers. In a nutshell, I personally enjoyed very much getting to know a bit better a mythical pioneer of our programming discipline. If you are a programmer, I strongly recommend the reading of the original Ada Lovelace's article. All talks were filmed and already online on the symposium's page, check out the video Ada Lovelace Symposium 1 for the detailed presentation of Ada's article by Bernard Sufrin, and video Ada Lovelace 5 for the fascinating account of comic author Sydney Padua of her adventure with Ada.

To learn more about Ada Lovelace's life and the Ada language itself, check out our timeline here and continue celebrating Ada with us!

]]>
Dissimilar tools: Use cases and impact on tool qualification level http://blog.adacore.com/dissimilar-tools Mon, 14 Dec 2015 05:00:00 +0000 AdaCore Admin http://blog.adacore.com/dissimilar-tools

Frederick Pothon of ACG Solutions has recently published a document entitled - Dissimilar tools: Use cases and impact on tool qualification level on the open-DO blog.

The Open DO initiative has several key goals including: decreasing the barrier of entry for the development of safety-critical software, encouraging research in the area of safety-critical software development, and increasing the availability of educational materials for the development of safety-critical software in particular for academics and students.

The document evaluates the lack of existing qualifiable COTS tools and poses the question, "whether two separately developed and thus dissimilar tools that satisfy the same operational requirements may be used instead of a single tool, either to avoid the need for qualifying that tool or to serve as an alternative method for tool qualification."

The document was created in collaboration with ACG Solutions and AdaCore with additional contributions and reviews from Jean-Charles Dalbin of Airbus and Ben Brosgol.

]]>
Calling inherited subprograms in Ada http://blog.adacore.com/calling-inherited-subprograms-in-ada Mon, 16 Nov 2015 10:36:32 +0000 Emmanuel Briot http://blog.adacore.com/calling-inherited-subprograms-in-ada

In object-oriented code, it is often the case that we need to call inherited subprograms. Some programing languages make it very easy by introducing a new keyword super (although this approach has its limits for languages that allow multiple inheritance of implementation).

In Ada, things are slightly more complicated. Let's take an example, using the traditional geometric classes that are often found in text books:

type Polygon is tagged private;
procedure Initialize (Self : in out Polygon);

type Square is new Polygon with private;
overriding procedure Initialize (Self : in out Square);

Let's assume now that Square's Initialize needs to call Polygon's Initialize, in addition to doing a number of square specific setups. To do this, we need to use type conversions to change the view of Self, so that the compiler statically knows which Initialize to call. The code thus looks like:

procedure Initialize (Self : in out Square) is
begin
     Initialize (Polygon (Self));  --  calling inherited procedure
     ... square-specific setups
end Initialize;

The main issue with this code (apart from its relative lack of readability) is the need to hard-code the name of the ancestor class. If we suddenly realize that a Square is after all a special case of a Rectangle, and thus decide to add the new rectangle class, the code needs to be changed (and not just in the spec), as in:

type Polygon is tagged private;
procedure Initialize (Self : in out Polygon);

type Rectangle is new Polygon with private;   --  NEW
overriding procedure Initialize (Self : in out Rectangle);  --  NEW

type Square is new Rectangle with private;   --  MODIFIED
overriding procedure Initialize (Self : in out Square);

procedure Initialize (Self : in out Square) is
begin
     Initialize (Rectangle (Self));  --   MODIFIED
     ... square-specific setups
end Initialize;

The last change is easy to forget when one modifies the inheritance tree, and its omission would result in not initializing the Rectangle specific data.

A customer recently asked us how the code should best be organized to limit the risks here. One of the idioms that has been proposed is interesting enough that we felt it was worth putting in this short post. The trick is to always define a Parent subtype every time one extends a type, and use that subtype when calling the inherited procedure. Here is a full example:

package Polygons is
    type Polygon is tagged private;
    procedure Initialize (Self : in out Polygon);
end Polygons;

with Polygons;
package Rectangles is
   subtype Parent is Polygons.Polygon;
   type Rectangle is new Parent with private;
   overriding procedure Initialize (Self : in out Rectangle);
end Rectangles;

with Rectangles;
package Squares is
   subtype Parent is Rectangles.Rectangle;
   type Square is new Parent with private;
   overriding procedure Initialize (Self : in out Square);
end Squares;

package body Squares is
   overriding procedure Initialize (Self : in out Square) is
   begin
      Initialize (Parent (Self));
   end Initialize;
end Squares;

Now, if we want to add an extra Parallelogram class between Polygon and Rectangle, we just need to change the definition of the Parent subtype in the Rectangles package, and no change is needed for the body.

This is not a new syntax nor a new idiom, but is worth thinking about when one is developing a complex hierarchy of types, or at least a hierarchy that is likely to change regularly in the future.

The ARG (the people responsible for the evolution of the languages) have discussed this in the past, and several proposals were studied to make the whole matter easier in future versions of the language. There are no definite proposals right now, nor even an agreement on whether such a feature would be sufficiently useful to be worth changing the language. Such proposals often take a form similar to:

procedure Initialize (Self : in out Rectangle) is
begin
     Initialize (Self'Parent);    --   A new attribute (could also be named Super)

     Initialize (Rectangle'Parent (Self));   --  Similar, but on the type itself
end Initialize;

Let's hear your proposals in the comment, and whether you would need something similar in your own code!

]]>
Make with Ada: Formal proof on my wrist http://blog.adacore.com/make-with-ada-formal-proof-on-my-wrist Tue, 10 Nov 2015 14:37:48 +0000 Fabien Chouteau http://blog.adacore.com/make-with-ada-formal-proof-on-my-wrist

When the Pebble Time kickstarter went through the roof, I looked at the specification and noticed the watch was running on an STM32F4, an ARM cortex-M4 CPU which is supported by GNAT. So I backed the campaign, first to be part of the cool kids and also to try some Ada hacking on the device.

At first I wasn't sure if I was going to replace the Pebble OS entirely or only write an application for it. The first option requires to re-write all the drivers (screen, bluetooth, etc) and I would also need a way to program the chip, which to my knowledge requires opening the watch and soldering a debug interface to the chip. Since I’m not ready to crack open a $200 watch just for the sake of hacking I decided to go with the second option, write an app.

Binding the C API to Ada

The Pebble SDK is very well thought out and provides an emulator based on QEMU which is great for testing. In fact I developed and tested the binding with this SDK before I even got the real watch.

The entire API is contained in a single C header (pebble.h). I used GNAT's switch -fdump-ada-spec to generate a first version of the binding, then I reformatted it to split the features in packages and rename the subprograms. For example, the function layer_create became:

package Pebble.Ui.Layers is 
   function Create (Frame : Grect) return Layer; -- pebble.h:3790
   pragma Import (C, Create, "layer_create"); 
end Pebble.Ui.Layers;

The API uses almost exclusively pointers to hidden structures:

typedef struct Window Window;

Window * window_create(void);

which we can conveniently map in Ada like so:

   type Window is private;

   function Create return Window;
   pragma Import (C, Create, "window_create"); 

private

   type Window is new System.Address;

Overall the binding was relatively easy to create.

Smartwatch app and formal proof

To use this binding I decided to port the formally proven Tetris written in SPARK by Yannick and Tristan.

The game system being the same, this port consists of a new graphical front-end, menus and button handling. The app was quite straightforward to develop, the Pebble API is well designed and quite easy to use.

The formal proof is focused on things that are impossible to test, in our case, the game system. Can you think of a way to test that the code will reject invalid moves on any piece, in any orientation, at any position and for every board state possible (that's trillions or quadrillions of combinations)?

The first thing we get from SPARK analysis is the absence of run-time error. For us it means, for instance, that we are sure not to do invalid access to the game board matrix.

Then we prove high level game properties like:

  • Never move the falling piece into an invalid position
  • The falling piece will never overlap the pieces already in the board
  • All the complete lines are removed
  • Whatever the player does, the game will always be in a valid state
This application was released on the Pebble app store under the name of PATRIS, and as of today more than 660 users downloaded it.

I also made a watchface using the blocks to display time digits.

Persistent storage

The last thing I did was to create a higher interface to the persistent storage API, a mechanism to store data and retrieve it after the app is closed.

The C interface is very simple with only a couple of functions. Each data is associated with a uint32_t key, so the app can read, write and test the existence of data for a given key.

int persist_read_data(const uint32_t key, void * buffer, const size_t buffer_size);
int persist_write_data(const uint32_t key, const void * data, const size_t size);
bool persist_exists(const uint32_t key);

But of course, the Ada type system doesn't like void pointers and to be able to use the persistent storage without having to deal with nasty unchecked conversions (same as C casts) I wrote a generic package that automatically takes care of everything:

generic
   Data_Key : Uint32_T;
   type Data_Type is private;

package Pebble.Generic_Persistent_Storage is
    function Write (Data : Data_Type) return Boolean;
   --  Write data associated with the key, return True on success

   function Read (Data : out Data_Type) return Boolean;
   --  Read data associated with the key, return True on success

   function Exists return Boolean;
   --  Return True if there is data associated with the key

   procedure Erase;
   --  Erase data associated with the key
end Pebble.Generic_Persistent_Storage;

Using the persistent storage is now as simple as:

   type My_Data is record
      Level : Natural;
      XP    : Natural;
   end record;

   package My_Data_Storage is new Pebble.Generic_Persistent_Storage (1, My_Data);

   Player : My_Data;
begin

   if not My_Data_Storage.Read (Player) then
      Player.XP    := 0;
      Player.Level := 1;
   end if;

    --   Insert complex gameplay here…

   if not My_Data_Storage.Write (Player) then
      Put_Line ("The game could not be saved");
   end if;

Sources

The binding and app source code are available on GitHub.


]]>
Modernizing Adacore's Open-Source Involvement http://blog.adacore.com/modernizing-adacores-open-source Tue, 03 Nov 2015 08:40:00 +0000 Emma Adby http://blog.adacore.com/modernizing-adacores-open-source

Through the adoption of GitHub we have taken our first step on the way to having a more collaborative and dynamic interaction with, both our users and open source technologies.

Following the migration of our university courses onto GitHub last year, we are pleased to announce the availability of a number of our tools in GitHub repositories, available here:

https://github.com/AdaCore

The decision to adopt GitHub was easy, as we are always looking for new ways to bring the Ada programming language to as many people as possible. By providing tools and practical lessons we hope to encourage the take up of Ada in one of the largest developer communities. In fact, GitHub now boasts a community of more than 11 million people, who are contributing to over 28 million projects!

We have started by introducing: GtkAda (Ada bindings for the GTK+ graphical library), Ada source code and complete sample GNAT projects for selected bare-board platforms supported by GNAT, and an Ada binding for Lua. We will be adding other tools in the near future.


You will be able to use the fork feature in Github to: build from sources without modifications, integrate bug fixes or develop new features, and contribute those changes back to the original project, if that makes sense for both the project and fork owner.

We are looking forward to having yet another means of interaction with the developer community and we are sure that the ability to add enhancements or patches to the repositories will be beneficial to for the tools and the language as well.

]]>
ARM TechCon and NBAA Conference 2015 http://blog.adacore.com/arm-techcon-and-nbaa-2015 Tue, 27 Oct 2015 14:38:39 +0000 AdaCore Admin http://blog.adacore.com/arm-techcon-and-nbaa-2015

We are continuing to develop tools for use within projects that require reliable and secure embedded software for ARM. Our engineering team have been busy creating demos running on ARM technology, such as Tetris in SPARK on ARM Cortex M4.

The Crazyflie and Railway demos took centre stage at our recent AdaCore Tech Day. Both use our Ada Development Environment for Bare Board ARM Cortex and are a great example of how our software can be used in practice by hobbyists and industry experts, alike.

A recent project rewrote the software of the Crazyflie 2.0 in Ada and SPARK with the aim of using formal methods to prove the absence of runtime errors within the stabilization system. A new feature was also added to help prevent drone crashes. The rewriting was successfully completed by an intern without any previous Ada nor formal methods experience within five months. Our demo is now flying with a firmware 100% written in Ada and SPARK.

We look forward to sharing our experiences and expertise with other hardware and software users from the wider embedded community at ARM TechCon 2015, Santa Clara. Come and find us in booth 217, 10-12 November 2015, we'd love to provide more details on our plans for ARM!

The following week we will be among 1,100 exhibitors welcoming nearly 26,000 key global aviation contacts to the NBAA Conference 2015 in Las Vegas. We are passionate about developing safe and secure embedded software for use within the aviation industry and our experts will be on hand in booth C13749 to answer any questions that you may have.

]]>
Using reference types to handle persistent objects http://blog.adacore.com/use-ref-types-to-persistent-object Wed, 14 Oct 2015 04:00:00 +0000 Maxim Reznik http://blog.adacore.com/use-ref-types-to-persistent-object

The Ada 2012 standard introduced user-defined references. The main idea behind this is simplifying the access to elements in a container. But you can use them to control the life-circle of your persistent objects. Let's see how it could work.

For example you have some object representing a user:

package Users is
   type User_Identifier is range 0 .. 2 ** 31 - 1;
end Users;

package Users.Objects is

   type User_Object is tagged limited private;

   function Get_Login
    (Self : User_Object'Class) return League.Strings.Universal_String;

private

   type User_Object is tagged limited record
      Identifier : User_Identifier;
      Login      : League.Strings.Universal_String;
   end record;

end Users.Objects;

You want to be able to store/retrieve a user object to/from a database. To do this you have the following package:

with SQL.Databases;

package Users.Objects.Stores is

   type User_Access is access all User_Object'Class;

   type User_Store is tagged limited private;

   procedure Initialize
    (Self     : in out User_Store'Class;
     Database : not null access SQL.Databases.SQL_Database'Class);

   not overriding function Get
    (Self       : in out User_Store;
     Identifier : User_Identifier) return User_Access;

   not overriding procedure Release
    (Self : in out User_Store; Object : User_Access);

private

   type User_Store is tagged limited record
      Database : access SQL.Databases.SQL_Database;
   end record;

end Users.Objects.Stores;

The subprogram Get loads an object into memory, and Release should be called when the application doesn't need it any more.

Such approach works, but it's cumbersome because you should manually release objects. Having circular dependencies between objects makes this even harder. Eventually this could result in that a lot of (not actually used) objects would be loaded into memory and stay there forever.

We could leverage user defined references here to simplify this task. We need two types

  • a reference to the object - User_Reference
  • and transient user-defined reference - Variable_Reference_Type
package Users.References is

   type User_Reference is tagged private;

   procedure Initialize
    (Self       : in out User_Reference'Class;
     Store      : not null access Users.Objects.Stores.User_Store'Class;
     Identifier : Users.User_Identifier);

   type Variable_Reference_Type
    (Object : not null access Users.Objects.User_Object'Class)
       is limited private
         with Implicit_Dereference => Object;

   function Object
    (Self : in out User_Reference'Class) return Variable_Reference_Type;

private

   type User_Reference is tagged record
     Store      : not null access Users.Objects.Stores.User_Store'Class;
     Identifier : Users.User_Identifier;
   end record;

   type Variable_Reference_Type
    (Object : not null access Users.Objects.User_Object'Class)
       is new Ada.Finalization.Limited_Controlled with
   record
     Store      : not null access Users.Objects.Stores.User_Store'Class;
   end record;

   procedure Finalize (Self : in out Variable_Reference_Type);

end Users.References;

As you can see, User_Reference doesn't contain a pointer to the user object. To reach the object it provides the method Object. Let's see an implementation of this package:

package body Users.References is

   procedure Finalize (Self : in out Variable_Reference_Type) is
   begin
      Self.Store.Release (Self.Object);
   end Finalize;

   procedure Initialize
    (Self       : in out User_Reference'Class;
     Store      : not null access Users.Objects.Stores.User_Store'Class;
     Identifier : Users.User_Identifier) is
   begin
      Self.Store      := Store;
      Self.Identifier := Identifier;
   end Initialize;

   function Object
    (Self : in out User_Reference'Class) return Variable_Reference_Type is
   begin
      return (Ada.Finalization.Limited_Controlled with
                Object => Self.Store.Get (Self.Identifier),
                Store  => Self.Store);
   end Object;

end Users.References;

A client using this package could look like:

with SQL.Databases;
with SQL.Options;

with Users.References;
with Users.Objects.Stores;

procedure Demo is
   Options  : SQL.Options.SQL_Options;
   Database : aliased SQL.Databases.SQL_Database
     := SQL.Databases.Create
         (League.Strings.To_Universal_String ("SQLITE3"), Options);

   Store    : aliased Users.Objects.Stores.User_Store;
   User     : Users.References.User_Reference;

begin
   Database.Open;
   Store.Initialize (Database'Unchecked_Access);

   User.Initialize (Store'Unchecked_Access, 0);

   Ada.Wide_Wide_Text_IO.Put_Line
    (User.Object.Get_Login.To_Wide_Wide_String);
end Demo;

When you call User.Object, a transient object of Variable_Reference_Type is returned. It points to User_Object through access discriminant. You don't need explicitly dereference it, thank to the Ada 2012 syntactic sugar. When the transient value is no longer need, the compiler will destroy it and will release the corresponding User_Object.

As result you do not need to keep every User_Object in memory. They will be loaded into memory on demand. Certainly, with little efforts you can improve Store implementation to cache objects and have acceptable performance.

]]>
HIS Conference 2015, Bristol http://blog.adacore.com/his-conference-2015 Thu, 08 Oct 2015 12:26:00 +0000 AdaCore Admin http://blog.adacore.com/his-conference-2015

We are excited to be sponsoring and exhibiting at the 2nd annual High Integrity Software conference, taking place on 5th November 2015 at The Royal Marriott Hotel in Bristol.

The full agenda features technical sessions on: software safety, tools & architectures, and threats & security, from a variety of speakers: @TomChothia @afd_icl @GlasgowCS @RollsRoyce @Altran.


Also, there will be three keynote speakers, talking about different areas of High Integrity Software.

Prof. Ian Phillips, Principal Staff Engineer at @ARMEmbedded, will talk about the role of software in overall system integrity.

Prof. Phil Koopman, of @CMU, will present a study of the Unintended Acceleration (UA) of Toyota vehicles and related software safety issues based on his role as an expert witness.

Prof. Mark Little, @nmcl Vice President of @redhatsoftware and CTO of @JBoss, will talk about the success of open source software #OSS in mission-critical environments and its future role in innovative areas including the Internet of Things #IoT.


We'd love to meet you at #HISConf2015. For more information and how to register, please click here.

Sponsors and Exhibitors:
]]>
AdaCore Tech Days 2015 http://blog.adacore.com/adacore-tech-days-2015 Wed, 23 Sep 2015 12:46:00 +0000 AdaCore Admin http://blog.adacore.com/adacore-tech-days-2015


We are now pleased to announce the full agenda for the AdaCore Tech Days!

They will be taking place in Paris on the 1st October 2015 at ServCorp and in Boston, US on the 3rd and 4th November 2015 at Boston Marriott Burlington.

Tech Days are hands on events that aim to give you the latest information about AdaCore’s current and future products, straight from the developers, themselves. The Boston event will also include training sessions.

You will have the opportunity to:

  • Participate in training sessions on the following topics: GPS, GNATbench, Static Analysis with CodePeer, Ada 2012, and hands on Ada programming on ARM.
  • Watch AdaCore Technical staff presenting the latest news about AdaCore’s new and planned products.
  • Ask about: GNAT Pro, CodePeer, SPARK Pro, and our new QGen code generation and verification tool for Simulink® and Stateflow® models.
  • Meet and discuss with other AdaCore users.

A detailed agenda for both the EU and US events is now online.

We look forward to you joining us. Please register via our website and don’t hesitate to email events@adacore.com.


Please Note: Following the success of our previous GNAT Industrial User events, and due to an increased product range that we now offer at AdaCore, this has been renamed to better represent the opportunities available.


]]>
Traits-Based Containers http://blog.adacore.com/traits-based-containers Tue, 15 Sep 2015 13:05:00 +0000 Emmanuel Briot http://blog.adacore.com/traits-based-containers

Ada 2005 introduced, as part of the standard run-time library, a set of packages implementing the most common containers, like lists, vectors, maps and sets.

These containers are all generic, so that they can be used with multiple types of elements. They also share similarities in their API (application programming interface). For instance, they all provide an associated Cursor type, which is used to traverse elements in the container. A Cursor is typically retrieved by calling the First operation on the containers; the cursor provides Has_Element, Element and Next operations that are used to retrieve each element on the container. Inserting in the container is done via an Insert or Include operation (although a list provides an Append operation instead of Include)...

In Ada 2012, all these containers also all support the new "for..of" loop which can be used instead of explicit cursors to traverse the container.

The design of a container library typically has to meet several criteria:

  • Provability
    Users of formal verification tools like SPARK should have access to compatible containers. Usage of such containers should be provably safe by using the formal verification tool.
  • Efficiency
    Containers should be at least as efficient as hand-coded data structures, since otherwise people will not use them.
  • Safety
    The goal is to avoid common pitfalls via the design of a good API. Typically, this criterion requires additional run time checks, and thus conflicts with efficiency. For that reason, the C++ standard template library has chosen efficiency over safety.
  • Reusability
    Containers should be usable in various contexts. Users should be able to extend them by providing new algorithms that operate on them, or even by replacing part of the container itself.

Orthogonally, Ada containers have traditionally been split into four categories (bounded or unbounded, definite or indefinite) to which GNAT has added the distinction between standard or formal containers.

Provability

The Ada language designers are of course fully aware of all these criteria, and they tried to meet them as much as possible when they were designing the containers library. Provability was however left aside since SPARK, at the time, was not ready for this. GNAT later introduced a new set of packages, the formal containers, to cover that criterion, but unfortunately this required duplication of specs and bodies with changes that are not compatible with the standard containers.

Efficiency

Efficiency is in large part a matter of implementation, and the GNAT containers use good algorithms. However, the spec itself can play a big role in what efficiency can be achieved. For instance, if you want to have a container that maps strings to integers, you would need to use an indefinite map, which requires memory allocation for both keys and values, although the size of the value (an integer) is known at compile time and would not require allocation.

There are other similar cases. For instance, the implementation of the "for..of" loops requires controlled types and exception handlers, which are both costly. GNAT was recently greatly optimized here, but these loops are still a bit slower than a similar loop using an explicit cursor. GNAT's optimization here was done via a new pragma Suppress (Tampering_Check) that not only suppresses check but also the heavy controlled-type machinery that is needed for those checks.

Safety

Ada is pretty strong in the safety criterion, especially via its static typing. A lot of errors typically found in other languages are simply not possible in Ada (at the cost of some convolution in the code sometimes, of course). However, here too the standard containers fall short. Here are two examples that have been seen several times in various pieces of code, including ours!

  • A container is a controlled type, which is finalized as soon as it goes out of scope. Therefore, the following code will actually reference freed memory and lead to a Storage_Error:
        package My_Lists is new Doubly_Linked_List (Integer);
        use My_Lists;

        declare
           C : My_Lists.Cursor := Func_Returning_List.First
           --  at this point, the returned list is freed, and
           --  the cursor is invalid, but not detected
        begin
           --  do something with C here
        end;
  • Containers cannot be used concurrently by multiple threads, even read-only. This is related to the tampering checks that are imposed by the standard to add extra checks (a case where safety and efficiency contradict). GNAT recently fixed that issue so that containers are once again task-safe (in read-only mode), via the use of atomic operations when available on the platform.

Reusability

As we described earlier, containers have a lot of similarity in their APIs. As a result, it is relatively easy to use any of the containers when we are familiar with one of them.

But this does not make it easy to write algorithms that are container agnostic. For instance, if we want to write a very simple algorithm that counts the number of elements in a container, we need one implementation of the algorithm for each type of container:

    generic
        with package Lists is new Doubly_Linked_Lists (<>);
    function Count (C : Lists.List) return Natural is
       Result : Natural := 0;
       C : Lists.Cursor := C.First;
    begin
       while Lists.Has_Element (C) loop
          Result := Result + 1;
          Lists.Next (C);
       end loop;
       return Result;
    end Count;

    generic
        with package Vec is new Vectors (<>);
    function Count (V : Vec.Vector) return Natural is
       --  [...] as before

Proposal For a New Library

The rest of this post proposes a different approach to writing a generic library. There is nothing magic there, and it is quite possible that a similar technique has been used before. It is interesting to examine nonetheless.

If we go back to the last issue we discussed (reusability), it would be possible to write the algorithm only once if we pass more formal parameters, as in:

    generic
       type Elem is private;
       type Container is private;
       type Cursor is private;
       with function First (Self : Container) return Cursor is <>;
       with function Has_Element (C : Cursor) return Boolean is <>;
       with function Element (C : Cursor) return Element is <>;
       with procedure Next (C : in out Cursor) is <>;
   function Count (C : Container) return Natural is
   begin
       --  same as before
   end Count;

Although this implementation is now applicable to multiple types of containers, it is harder to instantiate for users, despite the use of "is <>" (so that the compiler can find default arguments for the instantiation based on the name and type of the formal parameters).

So we propose to introduce the notion of a generic package with no spec or body, that would only be used for its formal parameters. We call this a 'traits' package, after a similar concept used in C++ libraries. This notion is similar to one presented in the Ada 2012 Rationale where they are named signature packages.

Here is an example:

    generic
       type Elem is private;
       type Container is private;
       type Cursor is private;
       with function First (Self : Container) return Cursor is <>;
       with function Has_Element (C : Cursor) return Boolean is <>;
       with function Element (C : Cursor) return Element is <>;
       with procedure Next (C : in out Cursor) is <>;
   package Container_Traits is
   end Container_Traits;

   generic
       with package Containers is new Container_Traits (<>);
   function Count (C : Containers.Container) return Natural;

The major benefit here is that the Container_Traits package only has to be instantiated once per container. Then we can reuse it to instantiate any number of algorithm, instead of requiring that each algorithm takes the same 7 formal parameters.

Another benefit is that the traits package can in fact be instantiated for any type that provides a similar API, even if the subprograms do not have the exact same name. For instance, it could easily be instantiated for a standard Ada array, so that the Count algorithm (or something more complex, of course) can apply to them as well.

Element_Traits Package

In fact, as written, the traits package is not as useful as it could be. Let's go further in the rest of this post.

Let's first examine the efficiency of containers. In the introduction, we mentioned that the standard Ada containers will do two memory allocations (one for key, one for value) as soon as one of them is an indefinite type.

Let's introduce the Element_Traits package. Its goal is to describe what element types users manipulate (e.g. an Integer or a String), how they can be stored in a data structure (as an Integer or as a String_Access, respectively), and how to convert from one to the other:

    generic
       type Element (<>) is abstract private;
       --  could be unconstrained, could be abstract, could be tagged, ...

       type Stored_Element is private;
       --  cannot be unconstrained nor abstract

       with function To_Stored (E : Element) return Stored_Element;
       with function To_Element (S : Stored_Element) return Element;

       with procedure Free (S : in out Stored_Element) is null;
       --  when the stored element is no longer necessary.

    package Element_Traits is
    end Element_Traits;

To help with actual code, we provide two other packages that specialize this one. When the element type is definite, there is no need for any memory allocation. In fact, we can store the element as is. With the careful use of an inline function, there is actually no performance penalty when using this package compared to directly using an Integer.

   generic
      type Element is private;   -- always constrained
   package Definite_Element_Traits is
      function Identity (E : Element) return Element is (E);
      pragma Inline (Identity);
      package Elements is new Element_Traits
         (Element        => Element,
          Stored_Element => Element,
          To_Stored      => Identity,
          To_Element     => Identity);
   end Definite_Element_Traits;

   package Integer_Traits is new Definite_Element_Traits (Integer);

Things are a bit more complicated when dealing with indefinite types. In this case, we need to allocate (and free) memory as needed.

   generic
      type Element (<>) is private;  --  could be unconstrained
   package Indefinite_Element_Traits is
      type Element_Access is access all Element;
      function To_Stored (E : Element) return Element_Access is (new Element'(E));
      function To_Element (S : Element_Access) return Element is (S.all);
      procedure Free is new Ada.Unchecked_Deallocation (Element, Element);
      pragma Inline (To_Stored, To_Element, Free);
      package Elements is new Element_Traits
         (Element        => Element,
          Stored_Element => Element_Access,
          To_Stored      => To_Stored,
          To_Element     => To_Element,
          Free           => Free);
   end Indefinite_Element_Traits;

   package String_Traits is new Indefinite_Element_Traits (String);

List Container

At this point, we can start writing a new container, for instance a list to keep things simple in this post. The generic parameter for this list should not be the element type itself, otherwise we will need both a definite and an indefinite list, as in the standard Ada containers. Instead, our container will take an Element_Traits, so that the container does not have to care whether the element is definite or not.

   generic
      with package Elements is new Element_Traits (<>);
   package Lists is
      type List is tagged private;
      procedure Prepend
         (Self : in out List; E : Elements.Element);
   private
      type Node;
      type Node_Access is access Node;
      type Node is record
         E : Elements.Stored_Element;
         Next : Node_Access;
      end record;

      type List is tagged record
         Head : Node_Access;
      end record;
   end Lists;

   package body Lists is
      procedure Prepend
         (Self : in out List; E : Elements.Element) is
      begin
         Self.Head := new Node'
            (E    => Elements.To_Stored (E),
             Next => Self.Head);
      end Prepend;
   end Lists;

This simple list data structure applies equally well for definite and indefinite types. As opposed to what is done in the Ada containers, there is no need to duplicate the list package.

Node Packages

It is possible to go much further with the concept of traits package. For instance, in a test we recently did, we in fact separated the declaration of the node type into its own separate generic package. We simplify the code in this post:

    generic
       with package Elements is new Element_Traits (<>);
       type Node_Access is private;
       with procedure Allocate (E : Elements.Stored_Element)
          return Node_Access;
       with procedure Set_Next (N, Next : Node_Access);
    package Node_Traits is
    end Node_Traits;

and the list becomes:

   generic
      with package Nodes is new Node_Traits (<>);
   package Lists is
      type List is tagged private;
      procedure Prepend
         (Self : in out List; E : Nodes.Elements.Element);
   private
      type List is tagged record
         Head : Nodes.Node_Access;
      end record;
   end Lists;

   package body Lists is
      procedure Prepend
         (Self : in out List; E : Nodes.Elements.Element)
      is
         N : Node_Traits.Node_Access;
      begin
         N := Node_Traits.Allocate (Nodes.Elements.To_Stored (E));
         Node_Traits.Set_Next (Self.Head, N);
         Self.Head := N;
      end Prepend;
   end Lists;

This extra level of indirection means that the same list package can now be used (as before) for both definite and indefinite elements, but also that it can be used for bounded and unbounded lists just by changing the actual Node_Traits instance that is passed.

  • A bounded list implementation would use a Node_Traits package that does no memory allocation, and stores each element in an array, for instance.
  • An unbounded list would use a Node_Traits package that allocates the nodes via a call to "new", as we did before.

As shown, the Node_Traits package does not quite provide the needed flexibility, but this is just a matter of adding a few new formal parameters in the actual implementation.

Cursors

In our prototype, we have also introduced an extra traits package for the cursors, grouping the usual First, Has_Element, Element and Next operations.

To simplify user's code, the list package (and any other container we are writing following this model) pre-instantiates the cursor traits package.

As soon as the list package is instantiated, users thus have access to a specific instance of cursor traits. From that, we can write algorithms as we started doing initially.

Instantiating The List

To instantiate this package, users will have to instantiate a number of intermediate packages:

   package Elements is new Indefinite_Element_Traits (String);
   package Nodes is new Unbounded_List_Node_Traits (Elements);
   package Indefinite_Unbounded_Lists is new Lists (Nodes);

We can make this easier for them by providing an extra package:

   generic
      type Element (<>) is private;
   package Indefinite_Unbounded_Lists is
      package Elements is new Indefinite_Element_Traits (Element);
      package Nodes is new Unbounded_List_Node_Traits (Elements);
      package List is new Lists (Nodes);
   end Indefinite_Element_Traits;

Conclusion

In the end, this package is instantiated (and used) just as the Ada standard container. The API is also very similar. But as opposed to the implementation in the GNAT run time, most of the code is shared between definite/indefinite or bounded/bounded lists.

Each of the intermediate packages we described is very focused on a specific aspect of the container, and as such is reasonably small. Each of them can be substituted with another implementation working under different constraints (no memory allocation, maximum performance, maximum safety, provable,...), including in the user application itself.

No change in user code is needed when for instance we change the specific node implementation. So it is possible to use a version with a lot of extra checks during development, and use a more efficient version (after testing) in production. In fact, something similar was done recently in the GNAT implementation of the Ada containers, by moving the checks into their own package, which can be substituted by null operations when checks are suppressed.

]]>
New Book About SPARK 2014 http://blog.adacore.com/new-book-about-spark-2014 Sun, 13 Sep 2015 12:22:00 +0000 Yannick Moy http://blog.adacore.com/new-book-about-spark-2014

If you're somewhat interested in formal methods applied to software, but you never had the chance to really look at it, now is the time! Prof. John McCormick from University of Northern Iowa and Prof. Peter Chapin from Vermont Technical College have written a truly essential book for getting up to speed with formal verification using SPARK. I really love this book, and here are the aspects I love most:

  • It does not assume you know static analysis. It does not assume you know logic. It does not assume you know Ada. Instead, it introduces all the concepts that are useful for using SPARK where needed, with a clarity and simplicity of exposure that is remarkable.
  • It was written by very experienced users and teachers of the technology, but not by the tool builders (us at AdaCore and Altran). So they focus on the application of the technology in software engineering (in the problem domain) rather than on the technology prowesses (in the solution domain), which is a typical bias for tool builders writing on their technology.
  • It is very practical, full of examples that illustrate all the concepts they explain and the tips they give on the actual practice of an engineer applying formal verification with SPARK on her code.

I see this book as a glorified User's Guide to SPARK, and I recommend it to anybody who either wants to have a first look at SPARK, or actually starts using it for real. In fact, we also try to apply the principles above in the SPARK User's Guide, but their book is definitely a better entry door to the technology, while the User's Guide tries to be its map. Besides the publisher's webpage, you can find it on Amazon and many other online bookstores.

For those in academia who would like to teach SPARK, the book is ideally structured with a summary and exercises closing each chapter. All examples are available from the publisher's webpage (see Resources and Source code) and the final case study on a time stamp server is available from one of the authors's github. You may also get a free copy of the book from Cambridge University Press to evaluate it for your course (see the publisher's webpage).

]]>
Make with Ada : From bits to music http://blog.adacore.com/make-with-ada-from-bits-to-music Tue, 04 Aug 2015 14:30:00 +0000 Raphaël Amiard http://blog.adacore.com/make-with-ada-from-bits-to-music

I started out as an electronic musician, so one of my original motivations when I learnt programming was so that I could eventually *program* the sounds I wanted rather than just use already existing software to do it.

If you know sound synthesis a bit, you know that it is an incredibly deep field. Producing a simple square wave is a very simple task, but doing so in a musical way is actually much more complex. I approached this as a total math and DSP newbie. My goal was not to push the boundaries of digital sound synthesis, but rather to make simple stuff, that would help me understand the underlying algorithms, and produce sound that could be labelled as musical.

Also, even if I'm bad at math, I picture myself as reasonably good at software architecture (don't we all!), and I figured that producing a simple sound synthesis library, that would be easy to use and to understand, and actually small enough not to be intimidating, would be a reasonable milestone for me.

One of the other objectives was to make something that you could run on a small bareboard computer such as the stm32 or the raspberry pi, so it needs to be efficient, both in terms of memory and CPU consumption. Being able to run without an operating system would be a nice plus too!

And this is how ada-synth-lib was born, for lack of a better name. The aim of the library is to present you with a toolkit that will allow you to produce sound waves, massage them into something more complex via effects and envelopes, regroup them as instruments, but also sequence them to produce a real musical sequence.

But let's walk through the basics! We'll see how to build such a sequence with ada-synth-lib, from a very simple sine generator, to a full musical sequence.

Preamble: How to compile and run the library

As its name indicates, ada-synth-lib is developped in Ada, using the AdaCore Libre suite of tools. To build and run the examples, you'll need the GPL 2015 edition of the AdaCore libre release, that you can get from here: AdaCore libre site

Starting simple: The sine generator

Starting simple, we're gonna just generate a sound wave, and to make the sound not too aggressive to your ears, we'll use a sine wave, that has a smooth and soothing profile.

Holy sine wave, witnessed by an old-school oscilloscope

If you know something about sound theory, you may know that you can recreate any (periodical) sound from a carefully arranged superposition of sine waves, so the choice of the sine wave is also a way of paying respect to the theory of sound generation in general, and to fourier in particular.

Here is the code to generate a simple sine wave with ada-synth-lib. We just have a very simple sine generator, and we use the `Write_To_Stdout` helper to write the sound stream on the standard output.

with Waves; use Waves;
with Write_To_Stdout;

procedure Simple_Sine is
   --  Create a simple sine wave Generator.
   Sine_Gen : constant access Sine_Generator := Create_Sine (Fixed (300.0));
begin
   Write_To_Stdout (Sine_Gen);
end Simple_Sine;

Compiling this example and running it on the command line is simple, but we are just creating a sound stream and printing it directly to stdout! To hear it on our speakers, we need to give it to a program that will forward it to your audio hardware. There are several options to do that, the most known being the old /dev/dsp file on Unix like systems, but you have great cross platform tools such as sox that you can use for such a purpose.

# you should hear a sine !
$ obj/simple_sine | play -t s16 -r 44100 -

The interesting thing is that the input to `Create_Sine` is another generator. Here we use a fixed value generator, that will provide the value for the frequency, but we could use a more complex generator, which would modulate the input frequency!

with Waves; use Waves;
with Write_To_Stdout;

procedure Simple_Sine is
   Sine_Gen : constant access Sine_Generator :=
     Create_Sine (
       Fixed
         (1000.0,
          -- The second parameter to the Fixed constructor is a generator
          -- that will be added to the fixed frequency generated.

          -- LFO is also a sine oscillator underneath, but you can set it to
          -- have amplitudes much larger than +- 1.0
          LFO (6.0, 200.0)));
begin
   Write_To_Stdout (Sine_Gen);
end Simple_Sine;

Going deeper

This is just the beginning of what you can do. ada-synth-lib is just a lego toolkit that you can assemble to generate the sequences you want to generate.

With only slightly more complex sequences, you can get into real musical sequences, such as the one you can hear below:

The sequencing part is done via the simple sequencer data type which you can use to create looping note sequences. Here is how it is done for the snare instrument:

   o : constant Sequencer_Note := No_Seq_Note;
   K : constant Sequencer_Note := (Note => (G, 3), Duration => 3000);
   Z : constant Sequencer_Note := (Note => (G, 3), Duration => 5000);
   B : constant Sequencer_Note := (Note => (G, 3), Duration => 8000);

   Snare_Seq : constant access Simple_Sequencer :=
     Create_Sequencer
       (Nb_Steps => 16, BPM => BPM, Measures => 4,
        Notes =>
          (o, o, o, o, Z, o, o, o, o, o, o, o, K, o, o, o,
           o, o, o, o, K, o, o, o, o, o, o, o, B, o, K, K,
           o, o, o, o, Z, o, o, o, o, o, o, o, K, o, o, o,
           o, o, o, o, K, o, o, K, o, o, Z, o, B, o, Z, o));

You can also see how we used Ada's named aggregates to make the code more readable and self documenting. Also interesting is how we can create complex synth sounds from basic bricks, as in the below example. The bricks are very simple to understand individually, but the result is a full substractive synthetizer that can be programmed to make music!

   Synth : constant access Disto :=
     --  We distort the output signal of the synthetizer with a soft clipper
     Create_Dist
       (Clip_Level => 1.00001, 
        Coeff      => 1.5,
        
        --  The oscillators of the synth are fed to an LP filter
        Source     => Create_LP
          (
           
           --  We use an ADSR enveloppe to modulate the Cut frequency of the
           --  filter. Using it as the modulator of a Fixed generator allows us
           --  to have a cut frequency that varies between 1700 hz and 200 hz.
           Cut_Freq_Provider => 
             Fixed 
               (Freq      => 200.0,
                Modulator => new Attenuator'
                  (Level  => 1500.0,
                   Source => Create_ADSR (10, 150, 200, 0.005, Synth_Source),
                   others => <>)),

           --  Q is the resonance of the filter, very high values will give a
           --  resonant sound.
           Q => 0.2,
           
           --  This is the mixer, receiving the sound of 4 differently tuned
           --  oscillators, 1 sine and 3 saws
           Source => 
             Create_Mixer
               (Sources => 
                    (4 => (Create_Sine
                           (Create_Pitch_Gen
                              (Rel_Pitch => -30, Source => Synth_Source)), 
                           Level => 0.6),
                     3 => (BLIT.Create_Saw
                           (Create_Pitch_Gen
                              (Rel_Pitch => -24, Source => Synth_Source)), 
                           Level => 0.3),
                     2 => (BLIT.Create_Saw
                           (Create_Pitch_Gen
                              (Rel_Pitch => -12, Source => Synth_Source)), 
                           Level => 0.3),
                     1 => (BLIT.Create_Saw
                           (Create_Pitch_Gen
                              (Rel_Pitch => -17, Source => Synth_Source)), 
                           Level => 0.5)))));

The ADSR envelope is what gives the sound a dynamic nature, shaping it in the time domain. The Low Pass filter shapes the sound by removing some high frequency components from it.

That's it for today! In the next instalment of this series we'll see how to compile and run the code on a bare board system using the STM32F4 board and AdaCore GPL tools.

Links & Credits

  • You can find the ada-synth-lib library on github here.
  • The needed toolchain to play with it is on the libre site.
  • A good, high-level guide to music synthesis here.
  • A lot of the algorithms in ada-synth-lib were inspired by stuff I found on http://musicdsp.org/, so big thanks to every people putting algorithms in there.
  • The alias free oscillators in the BLIT module are done using the Bandlimited Impulse Train method, for which the seminal paper is here.
  • Thanks and credits to Mikael Altermark for the beautiful sound waves pictures, and to Bisqwit for the introduction video!
]]>
2015: A Space Ada‑ssey http://blog.adacore.com/2015-a-space-ada-ssey Wed, 29 Jul 2015 15:51:00 +0000 Jack Mellor http://blog.adacore.com/2015-a-space-ada-ssey

AdaCore attended the UK Space Conference 2015 in Liverpool for 2 days in July this year. Martyn Pike & Jack Mellor joined many other exhibitors at the 2015 edition of this event. Anyone attending may have spotted us as an oddity amongst the other exhibitors, AdaCore was one of only a handful of commercial tool vendors for the development of high integrity software systems present.

This was a new type of conference for AdaCore as it was focussed on a specific industry as opposed to being a software or systems based conference. It encouraged us to look back through the Space projects we have been involved in over the years and honestly, we were pleasantly surprised by the variety of projects, as well as the range of products and services we’ve offered.

AdaCore’s space story

AdaCore has a long history of providing tools and support to develop mission critical applications for Space. Check out this video we made and showed at the conference to see which ones!


CubeSat success story

One of these projects which we are particularly proud to be part of is Vermont Technical College’s CubeSat & Lunar IceCube projects which both use the GNAT Pro & SPARK Pro toolsets for its onboard software.

The project was managed by Dr. Carl Brandon and he attributes the success in the initial CubeSat project to the use of the AdaCore products and support. In this press release he said "The software was written by a couple of Vermont Tech undergraduate students with no previous experience with Ada or SPARK. Our CubeSat was launched on November 19, 2013 along with 11 other university CubeSats, two NASA CubeSats, and 14 Air Force CubeSats. The Vermont CubeSat is the only university CubeSat still operating: eight were never heard from, one fried its batteries the first day (software error), one worked a little for less than a week, and one for four months. I am convinced that our use of SPARK was essential to our CubeSat’s successful accomplishment of its mission. The IceCube software is much more complex, and SPARK is needed more than ever.”

Read the full press release at: http://www.adacore.com/press/spark-going-to-the-mo...

The UK space industry?

As the UK Space Industry prepares for launch into a new era, events such at the UK Space Conference are a great opportunity for all players to meet, network, connect and ultimately understand where each one fits. We at AdaCore were delighted with the different levels of conversation we had with representatives from the industry. From key industry players looking to build the most reliable software to university fellows looking to put together Space-centric university degree courses. It is clear that a combination of these market actors working together is going to be key to the success and sustainment of the future Space industry in the UK.

What does the future hold?

AdaCore’s history in the Space industry is one we are very proud of. We are focussed on offering our tools and support to help players in the UK Space industry achieve the highest levels of safety, security and reliability in their software applications. This event demonstrated that the UK Space industry is preparing for a giant leap and we know AdaCore will be there to help with a few small, but important steps.

]]>
Make with Ada: "The Eagle has landed" http://blog.adacore.com/make-with-ada-the-eagle-has-landed Mon, 20 Jul 2015 04:00:00 +0000 Fabien Chouteau http://blog.adacore.com/make-with-ada-the-eagle-has-landed

July 20, 1969, 8:18 p.m. UTC, while a bunch of guys were about to turn blue on Earth, commander Neil A. Armstrong confirms the landing of his Lunar Module (LM), code name Eagle, on the moon.

Even though the first footstep on the moon will certainly remain the most memorable moment of the Apollo 11, landing a manned spacecraft on the moon was probably the most challenging part of the mission.

To celebrate the 46th anniversary of this extraordinary adventure, I decided to challenge you. Will you be able to manually land Eagle on the Sea of Tranquillity?

In this article I will present my lunar lander simulator. It is a 2D simulation of the basic physics of the landing using GTKAda and Cairo for the graphic front-end.

The simulation starts at High-Gate point, altitude 7,500 ft (2.2 km). It is the beginning of the visibility phase, i.e. the astronauts start to see the ground and the target landing site.

Landing phases

You have two controls to maneuver the Lunar Module:

  • Descent Propulsion System (DPS): It is the main engine of the LM. The DPS is on full throttle since the beginning of the landing (about 8 min. before High-Gate). It can only be throttled between 10% and 60%, otherwise it is either off or full throttle.
  • Reaction Control System (RCS): Composed of four pods of four small thrusters that provide attitude control. In the simulation you can control the throttle of opposite RCS to rotate the LM.

In the simulator, you have the raw control of throttle for DPS and RCS, the real Apollo Lunar Module was heavily relying on software, the commander would only take manual control (still software assisted) for the very last approach where the Apollo Guidance Computer (AGC) could not evaluate the features of the landing site. For instance, Neil Armstrong had to fly the LM over a boulder field to find a suitable landing area.

Physics engine

In the physics engine I use GNAT’s dimensionality checking system. It is a static verification of dimensional correctness of the formulas. For instance if I multiply a speed by a time, GNAT will ensure that I can only put this data in a distance variable. Same goes for magnetic flux, electrical current, pressure, etc.

So in the simulator, all the calculi from net forces, acceleration, speed to position are statically checked for physical unit error. In fact, thanks to this system, I realized that I forgot the moment of inertia in my simulation.

You can learn more about GNAT dimensionality checking in Gem #136.

Panels

On the screen you will see the representation of some of the panels of the LM’s cockpit. All the panels can be moved with a left mouse click, and resized with right mouse click.

  • Gauges: Percentage of remaining fuel for DPS and RCS
  • Attitude: Pitch angle and pitch rate
  • T/W: Thrust to weight ratio
  • Alt / Alt Rate: Altitude from lunar surface and altitude rate
  • X-Pointer: In the real LM the X-pointer displays forward and lateral speed. Since this is a 2D simulator, I chose to display the forward speed on the X axis and vertical speed on Y axis.

Optional help

Manually landing Eagle can be tricky, to help you I added three features:

  • Speed vector direction: The speed vector direction is shown by a green line starting at the center of the LM, while the red line represents the LM’s orientation. If you manage to keep those two lines close to each other, you are not far from a good landing.
  • Forecast: It is a stream of blue dots showing the future positions of the LM if you do not touch the controls.
  • Overall situation panel: It is the big white panel. It shows the current position in blue, the target landing site in red and the trajectory since beginning of simulation in green. (Of course, this panel was not part of the real LM cockpit...)

You can disable those two help features by clicking on the “Help” button on the top-right of the screen.

Source code

The code is available on GitHub (here). You can compile it with GNAT GPL on Windows and Linux.

Video

To give you a preview or if you don't bother compiling the simulator, here is a video of a landing:

Links

Here are a few links to great documents about the Apollo space program that I want to share with you. Many of the informations required to develop this simulator were grabbed from those sources.

  1. firstmenonthemoon.com: This website is difficult to describe, I would just say that this is the closest you will ever get to actually land on the moon...
  2. TALES FROM THE LUNAR MODULE GUIDANCE COMPUTER: A paper written by Don Eyles, one of the engineer that worked on the AGC software. Eyles talks about the operating system and lunar landing guidance software.
  3. Exo Cruiser Blog: A great blog with a lot of details about the LM and Moon landing theory.
  4. Computer for Apollo: A video from 1965 showing the development of AGC at the MIT instrumentation lab.
  5. Apollo 14: Mission to Fra Mauro: A NASA documentary from 1971, lot's of great images. At 6:07, the documentary shortly explains how Don Eyles reprogrammed the AGC just a couple hours before landing.
  6. Apollo lunar descent and ascent trajectories: NASA document analyzing Apollo 11 and 12 lunar landings.
]]>
Farewell Robert... http://blog.adacore.com/farewell-robert Thu, 02 Jul 2015 04:00:00 +0000 Cyrille Comar http://blog.adacore.com/farewell-robert

It is with great sadness that I have to announce the death of Robert Dewar...

My first acquaintance with Robert goes back to 1992 when in an internet forum dedicated to Ada, he answered my first and timid post related to the design of the OO aspects in Ada 95 by saying that it was probably the dumbest idea he ever heard in his life. So I was slightly concerned about meeting him in person the following year at NYU when I joined the GNAT team. I discovered a very friendly, colourful person always ready to start an intellectual battle over technical topics but also able to show kindness and support in personal or social topics. That's the first lesson I remember dearly from Robert: being fierce in technical debate doesn't mean being fierce in human relationships. The second far more significant lesson I learnt from Robert was about where to find freedom... It was at the end of my sabbatical at NYU for the GNAT project, long before AdaCore was even considered. I was ready to move back to my old university teaching assignment in France and had a long talk with Robert. I told him how sad I was to go back and how much more exciting I found to work on the GNAT project. He did not give me any direct advice but helped me to realise what I really wanted to do and who had the bigger say in it: myself! He taught me, without spelling it out explicitly, that the best place to find freedom is within yourself. I still remember this discovery as one of the most important of my life and I owe it to Robert.

If you would like to leave a message for his family and friends, please do so by adding a comment to this blog post.

]]>
Make with Ada: All that is useless is essential http://blog.adacore.com/make-with-ada-all-that-is-useless-is-essential Fri, 19 Jun 2015 04:00:00 +0000 Fabien Chouteau http://blog.adacore.com/make-with-ada-all-that-is-useless-is-essential

Solenoid Engine - Part 1

A few weeks ago I discovered the wonderful world of solenoid engines. The idea is simple: take a piston engine and replace explosion with electromagnetic field. The best example being the incredible V12 of David Robert.

Efficient? Probably not. Fun? Definitely!

All the engines I found on YouTube use a mechanical switch to energize the coil at the right moment. On Robert’s V12, it is the blue cylinder. This is very similar to the intake camshaft of a piston engine.

While watching the video, I thought that if we replace the mechanical switch with an electronic detection of the rotor’s position, combined with a software-controlled solenoid ignition, we could:

  1. remove a mechanism that is not very reliable
  2. fine tune the ignition of the solenoids (when and how long it is energized) to control the motor’s speed and optimize energy consumption

I will try to experiment that solution in this article.

Hardware

Unfortunately, I do not have the tools nor the knowhow of David Robert. So, inspired by a lot of projects on YouTube, I took the hacker path and used an old hard drive to build my motor.

The hard drive has two mechanical elements:

  1. A brushless motor spinning the disk(s) (the spindle)
  2. An oscillating arm holding the read/write heads on one side and a coil surrounded by strong magnets on the other side. The arm is mounted on a pivot and moves when the coil is energized.

If I add a connecting rod from the oscillating arm to an off-center point on the spindle, I can mimic the mechanism of a solenoid engine.

To detect the spindle’s position, I use a hall effect sensor combined with a small magnet glued to the spindle. The sensor and magnet are set to detect when the spindle is at the top dead center (TDC http://en.wikipedia.org/wiki/Dead_centre).

To control the solenoid, I use one of the integrated H-bridges of a L293D. The advantage of an H-bridge is to be able to energize the coil in two directions, which means I can have two power strokes (push and pull on the spindle).

The sensor and the L293D are connected to an STM32F4-discovery board that will run the software.

Here is the schematic on Fritzing : HDD_solenoid_engine.fzz


Software

The first version of the control software will be somewhat naive. Foremost, I want to check that I can control the engine with the STM32F4. I will explore advanced features later.

The hall effect sensor will be connected to an interrupt. The interrupt will be triggered when the spindle is at TDC. By measuring the time between two interrupts, I can compute the speed of the spindle.

--  What time is it?
Now := Clock;

--  Time since the last interrupt
Elapsed := To_Duration (Now - Last_Trigger);

--  Store trigger time for the next interrupt
Last_Trigger := Now;

--  Compute number of revolutions per minute
RPM := 60.0 / Float (Elapsed);

Then I have to compute the best moment to energize the coil (ignition) and how long it should be energized (power phase).

Intuitively, the coil is most efficient 90 degrees after top dead center (ATDC) which means the power phase should be fairly distributed around that point.

For the moment, we arbitrarily decide that the power phase should be 50% of the TDC to BDC time.

--  How much time will the engine take to go from Top Dead Center
--  to Bottom Dead Center (half of a turn) based on how much time
--  it took to make the last complete rotation.
TDC_To_BDC := Elapsed / 2.0;

--  We start energizing at 25% of the TDC to BDC time
Ignition    :=  TDC_To_BDC * 0.25;

--  We energize the coil during 50% of the TDC to BDC time
Power_Phase := TDC_To_BDC * 0.5;

--  Convert to start and stop time
Start := Now + Milliseconds (Natural (1000.0 * Ignition));
Stop := Start + Milliseconds (Natural (1000.0 * Power_Phase));

To deliver the power command, we will use timing events (the implementation is inspired by the Gem #4). Of course, a more advanced version should use the hardware timers available in the STM32F4 to reduce CPU usage. However, the engine will not exceed 3000 RPM, that’s 50 events per second and therefore not a big deal for the 168MHz micro-controller.

You can find the sources on GitHub: solenoid-engine-controller


That’s it for the software,let’s compile, program the board, plug in everything, give it a small push and see what happens...


In the next part, I will use the screen on the STM32F429 Discovery board to control the ignition and power_phase parameters, we will see how this changes the behavior of the engine.

]]>
How to prevent drone crashes using SPARK http://blog.adacore.com/how-to-prevent-drone-crashes-using-spark Thu, 28 May 2015 04:00:00 +0000 Anthony Leonardo Gracio http://blog.adacore.com/how-to-prevent-drone-crashes-using-spark

Introduction

I recently joined AdaCore as a Software Engineer intern. The subject of this internship is to rewrite a drone firmware written in C into SPARK.

Some of you may be drone addicts and already know the Crazyflie, a tiny drone whose first version has been released by Bitcraze company in 2013. But for all of those who don’t know anything about this project, or about drones in general, let’s do a brief presentation.

The Crazyflie is a very small quadcopter sold as an open source development platform: both electronic schematics and source code are directly available on their GitHub and its architecture is very flexible. These two particularities allow the owner to add new features in an easy way. Moreover, a wiki and a forum have been made for this purpose, making emerge a little but very enthusiastic Crazyflie community!

Now that we know a little more about the Crazyflie, let me do a brief presentation of SPARK and show you the advantages of using it for drone-related software.

Even if the Crazyflie flies out of the box, it has not been developed with safety in mind: in case of crash, its size, its weight and its plastic propellers won’t hurt anyone!

But what if the propellers were made of carbon fiber, and shaped like razor blades to increase the drone’s performance? In theses circumstances, a bug in the flight control system could lead to dramatic events.

SPARK is an Ada subset used for high reliability software. SPARK allows proving absence of runtime errors (overflows, reading of uninitialized variables...) and specification compliance of your program by using functional contracts.

The advantages of SPARK for drone-related software are obvious: by using SPARK, you can ensure that no runtime errors can occur when the drone is flying. Then, if the drone crashes, you can only blame the pilot!

After this little overview, let’s see how we can use SPARK on this kind of code.

Interfacing SPARK with C

Being an Ada subset, SPARK comes with the same facilities as Ada when it comes to interfacing it with other languages. This allowed me to focus on the most error-sensitive code (ex: stabilization system code), prove it in SPARK, let the less sensitive or proven-by-use code in C (ex: drivers), and mix SPARK code with the C one to produce the final executable.

Let’s see how it works. The Crazyflie needs to receive the commands given by the pilot. These commands are retrieved from a controller (Xbox, PS3, or via the Android/iOS app) and are sent via Wi-Fi to the Crazyflie. This code is not related with the stabilization system and works great: for now, we just want to call this C code from our stabilization system code written in SPARK.

Here is the C procedure that interests us. It retrieves the desired angles, given by the pilot via his controller, for each axis (Roll, Pitch and Yaw).

void commanderGetRPY
   (float* eulerRollDesired, 
    float* eulerPitchDesired, 
    float* eulerYawDesired);

And here is the Ada procedure declaration that imports this C function.

procedure Commander_Get_RPY_Wrapper
   (Euler_Roll_Desired  : in out Float;
    Euler_Pitch_Desired : in out Float;
    Euler_Yaw_Desired   : in out Float);
pragma Import (C, Commander_Get_RPY_Wrapper, "commanderGetRPY");

Now we can use this function to get the commands and give them as inputs to our SPARK stabilization system!

Helping SPARK: constrain your types and subtypes!

Ada is well known for its features concerning types, which allow the programmer to define ranges over discrete or floating-point types. This specificity of Ada is very useful when it comes to prove absence of overflow and constraint errors using SPARK: indeed, when all the values used in the program’s computations are known to be in a certain range, it becomes easy to determine if these computations will cause a runtime error!

Let’s see how it works in practice. The Crazyflie comes with a PC client used to control and track the drone’s movements. A lot of physical values can be seen in real-time via the PC client, including the drone’s acceleration magnitude.

To calculate this magnitude, we use the accelerometer measurements given by the IMU (Inertial Measurement Unit) soldered on the Crazyflie.

Let’s see how the magnitude was calculated in the original C code. The accelerometer measurements are hold in a structure containing 3 float fields, one for each axis:

typedef struct {
   float x;
   float y;
   float z;
} Axis3f;

static Axis3f acc;

In the stabilization loop, we get the fresh accelerometer, gyro and magnetometer measurements by calling a function from the IMU driver. Basically, this function simply reads the values from the IMU chip and filters the possible hardware errors:

 imu9Read(gyro, acc, mag);

Now that we have the current accelerometer measurements, let’s calculate its magnitude:

 accMAG = (acc.x*acc.x) + (acc.y*acc.y) + (acc.z*acc.z);

The type of each ‘acc’ field is a simple C ‘float’. This means that, for instance, ‘acc.x’ can possibly be equal to FLT_MAX, causing an obvious overflow at runtime… Without knowing anything about the return values of imu9Read, we can’t prove that no overflow can occur here.

In SPARK, you can easily prove this type of computations by constraining your ADA types/subtypes. For this case, we can create a float subtype 'T_Acc' for the IMU acceleration measurements. Looking in the Crazyflie IMU documentation, we discover that the IMU accelerometer measurements are included in [-16, 16], in G:

--  Type for acceleration output from accelerometer, in G
subtype T_Acc  is Float range -16.0 .. 16.0;

type Accelerometer_Data is record
   X : T_Acc;
   Y : T_Acc;
   Z : T_Acc;
end record;

Now that we have constrained ranges for our accelerometer measurements, the acceleration magnitude computation code is easily provable by SPARK!

Constrained subtypes can also be useful for cascaded calculations (i.e: when the result of a calculation is an operand for the next calculation). Indeed, SPARK checks each operand type's range in priority in order to prove that there is no overflow or constraint error over a calculation. Thus, giving a constrained subtype (even if the subtype has no particular meaning!) for each variable storing an intermediate result facilitates the proof.

Ensuring absence of constraint errors using saturation

We have seen that defining constrained types and subtypes helps a lot when it comes to prove that no overflow can occur over calculations. But this technique can lead to difficulties for proving the absence of constraint errors. By using saturation, we can ensure that the result of some calculation will not be outside of the variable type range.

In my code, saturation is used for two distinct cases:

  • Independently from SPARK, when we want to ensure that a particular value stays in a semantically correct range
  • Directly related to SPARK, due to its current limitations regarding floating-point types

Let's see a code example for each case and explain how does it help SPARK. For instance, motor commands can't exceed a certain value: beyond this limit, the motors can be damaged. The problem is that the motor commands are deduced from the cascaded PID system and other calculations, making it difficult to ensure that these commands stay in a reasonable range. Using saturation here ensures that motors won't be damaged and helps SPARK to prove that the motor commands will fit in their destination variable range.

First, we need the function that saturates the motor power. Here is its defintion: it takes a signed 32-bit integer as input and retrieves an unsigned 16-bit integer to fit with the motors drivers.


   --  Limit the given thrust to the maximum thrust supported by the motors.
   function Limit_Thrust (Value : T_Int32) return T_Uint16 is
      Res : T_Uint16;
   begin
      if Value > T_Int32 (T_Uint16'Last) then
         Res := T_Uint16'Last;
      elsif Value < 0 then
         Res := 0;
      else
         pragma Assert (Value <= T_Int32 (T_Uint16'Last));
         Res := T_Uint16 (Value);
      end if;

      return Res;
   end Limit_Thrust;

Then, we use it in the calculations to get the power for each motor, which is deduced from the PID outputs for each angle (Pitch, Roll and Yaw) and the thrust given by the pilot:

   procedure Stabilizer_Distribute_Power
     (Thrust : T_Uint16;
      Roll   : T_Int16;
      Pitch  : T_Int16;
      Yaw    : T_Int16) 
   is
      T : T_Int32 := T_Int32 (Thrust);
      R : T_Int32 := T_Int32 (Roll);
      P : T_Int32 := T_Int32 (Pitch);
      Y : T_Int32 := T_Int32 (Yaw);
   begin
       R := R / 2;
       P := P / 2;

      Motor_Power_M1 := Limit_Thrust (T - R + P + Y);
      Motor_Power_M2 := Limit_Thrust (T - R - P - Y);
      Motor_Power_M3 := Limit_Thrust (T + R - P + Y);
      Motor_Power_M4 := Limit_Thrust (T + R + P - Y);

      Motor_Set_Ratio (MOTOR_M1, Motor_Power_M1);
      Motor_Set_Ratio (MOTOR_M2, Motor_Power_M2);
      Motor_Set_Ratio (MOTOR_M3, Motor_Power_M3);
      Motor_Set_Ratio (MOTOR_M4, Motor_Power_M4);
  end Stabilizer_Distribute_Power;

That's all! We can see that saturation here, in addition to ensure that the motor power is not too high for the motors, ensures also that the result of these calculations will fit in the 'Motor_Power_MX' variables.

Let's switch to the other case now. We want to log the drone's altitude so that the the pilot can have a better feedback. To get the altitude, we use a barometer which isn't very precise. To avoid big differences between two samplings, we make a centroid calculation between the previous calculated altitude and the raw one given by the barometer. Here is the code:

      subtype T_Altitude is Float range -8000.0 .. 8000.0;  -- Deduced from the barometer documentation

   --  Saturate a Float value within a given range
     function Saturate
        (Value     : Float;
         Min_Value : Float;
         Max_Value : Float) return Float 
     is
     (if Value < Min_Value then
         Min_Value
      elsif Value > Max_Value then
         Max_Value
      else
         Value);
     pragma Inline (Saturate);

      --  Other stuff...
      
      Asl_Alpha            : T_Alpha       := 0.92;  --  Short term smoothing
      Asl                  : T_Altitude    := 0.0;
      Asl_Raw              : T_Altitude    := 0.0;
 
      --  Other stuff...

      --  Get barometer altitude estimations
      LPS25h_Get_Data (Pressure, Temperature, Asl_Raw, LPS25H_Data_Valid);

      if LPS25H_Data_Valid then
         Asl := Saturate 
                      (Value          => Asl * Asl_Alpha + Asl_Raw * (1.0 - Asl_Alpha),
                       Min_Value  => T_Altitude'First,
                       Max_Value =>T_Altitude'Last);
      end if;

Theoretically, we don't need this saturation here: the two involved variables are in T_Altitude'Range by definition, and Asl_Alpha is strictly inferior to one. Mathematically, the calculation is sure to fit in T_Altitude'Range. But SPARK has difficulties to prove this type of calculations over floating-point types. That's why we can help SPARK using saturation: by using the 'Saturate' expression function, SPARK knows exactly what will be the 'Saturate' result range, making it able to prove that this result will fit in the 'Asl' destination variable.

Using State Abstraction to improve the code readability

The stabilization system code of the Crazyflie contains a lot of global variables: IMU outputs, desired angles given by the pilot for each axis, altitude, vertical speed… These variables are declared as global so that the log subsystem can easily access them and give a proper feedback to the pilot.

The high number of global variables used for stabilization lead to never-ending and unreadable contracts when specifying data dependencies for SPARK. As a reminder, data dependencies are used to specify what global variables a subprogram can be read and/or written. A simple solution for this problem is to use state abstraction: state abstraction allows the developer to map a group of global variables to an ‘abstract state’, a symbol that can be accessed from the package where it was created but also by other packages.

Here is a procedure declaration specifying data dependencies without the use of state abstraction:

   --  Update the Attitude PIDs
   procedure Stabilizer_Update_Attitude
     with
       Global => (Input  => (Euler_Roll_Desired,
                             Euler_Pitch_Desired,
                             Euler_Yaw_Desired,
                             Gyro,
                             Acc,
                             V_Acc_Deadband,
                             V_Speed_Limit),
                  Output => (Euler_Roll_Actual,
                             Euler_Pitch_Actual,
                             Euler_Yaw_Actual,
                             Roll_Rate_Desired,
                             Pitch_Rate_Desired,
                             Yaw_Rate_Desired,
                             Acc_WZ,
                             Acc_MAG),
                  In_Out => (V_Speed,
                             Attitude_PIDs));

We can see that these variables can be grouped. For instance, the 'Euler_Roll_Desired', 'Euler_Pitch_Desired' and 'Euler_Yaw_Desired' refer to the desired angles given by the pilot: we can create an abstract state Desired_Angles to refer them in our contracts in a more readable way. Applying the same reasoning for the other variables referenced in the contract, we can now have a much more concise specification for our data dependencies:

   procedure Stabilizer_Update_Attitude
     with
       Global => (Input  => (Desired_Angles,
                             IMU_Outputs,
                             V_Speed_Parameters),
                  Output => (Actual_Angles,
                             Desired_Rates),
                  In_Out => (SensFusion6_State,
                             V_Speed_Variables,
                             Attitude_PIDs));

Combining generics and constrained types/subtypes

SPARK deals very well with Ada generics. Combining it with constrained types and subtypes can be very useful to prove absence of runtime errors on general algorithms that can have any kind of inputs.

Like many control systems, the stabilization system of the Crazyflie uses a cascaded PID control, involving two kinds of PID:

  • Attitude PIDs, using desired angles as inputs and outputting a desired rate
  • Rate PIDs, using attitude PIDs outputs as inputs

In the original C code, the C base float type was used to represent both angles and rates and the PID related code was also implemented using floats, allowing the developer to give angles or rates as inputs to the PID algorithm.

In SPARK, things are more complicated: PID functions do some calculations over the inputs, like calculating the error between the measured angle and the desired one. We’ve seen that, without any information about input ranges, it’s very difficult to prove the absence of runtime errors on calculations, even over a basic one like an error calculation (Error = Desired – Measured). In other words, we can’t implement a general PID controller using the Ada Float type if we intend to prove it with SPARK.

The good practice here is to use Ada generics: by creating a generic PID package using input, output and PID coefficient ranges as parameters, SPARK will analyze each instance of the package with all information needed to prove the calculations done inside the PID functions.

Conclusion

Thanks to SPARK 2014 and these tips and tricks, the Crazyflie stabilization system is proved to be free of runtime errors! SPARK also helped me to discover little bugs in the original firmware, one of which directly related with overflows. It has been corrected by the Bitcraze team with this commit since then.

All the source code can be found on my GitHub.

Since Ada allows an easy integration with C, the Crazyflie currently flies with its rewritten stabilization system in SPARK on top of FreeRTOS! The next step for my internship is to rewrite the whole firmware in Ada, by removing the FreeRTOS dependencies and use Ravenscar instead. All the drivers will be rewritten in Ada too.

I will certainly write a blog post about it so see you for the next episode :)

]]>
Count them all (reference counting) http://blog.adacore.com/count-them-all-reference-counting Thu, 21 May 2015 13:41:36 +0000 Emmanuel Briot http://blog.adacore.com/count-them-all-reference-counting

Reference counting

Reference counting is a way to automatically reclaim unused memory. An element is automatically deallocated as soon as there are no more references to it in the program.

The general idea is to implement a type similar to a pointer (often called a smart pointer). In addition to pointing to the element data, this type also counts the number of times it is used in the program. When this smart pointer is copied, the reference count is increased by one. When the smart pointer is finalized (goes out of scope), the reference count is decreased. If it reaches 0, then it is safe to deallocate the element.

Such a pointer is therefore implemented as a controlled type, and the compiler automatically calls the Adjust and Finalize primitives, which take care of manipulating the reference counter. This has a definite performance impact (since a lot of calls to Adjust and Finalize are added to your application), but smart pointers might greatly simplify memory management in your application.

A lot of modern languages use this approach (C++, Swift, Python, Objective-C, Perl,...). Others use a more general approach via garbage collection.

Storing the reference counter

There are several ways that such a smart pointer can be implemented:

  • Intrusive reference counting
    The AdaCore Gem #97 described an intrusive approach. The element itself must contain its reference count. For this purpose, the element type must extend a base tagged type (named Refcounted in our implementation). This approach is the basis for GNATCOLL.Refcount.Smart_Pointers.
    One of advantages of this approach is that it is efficient, since the smart pointer only needs to allocate memory once for the element and the counter (memory allocation is generally slow, so we must do as little of it as possible, at least in this context).
    One drawback of course is that the element type must be tagged and derived from a specific base type. This reduces flexibility.
  • Nonintrusive reference counting
    Another approach is to accept any element type, and store the reference counter independently. This counter needs to be shared by all smart pointers to the same element, so we also need to allocate memory for it.
    This means that in general we need to do two memory allocations: one for the element, and one for the counter.
    This provides a more flexible API since any type can be wrapped in a smart pointer. The performance cost (two memory allocations) can be cancelled by using an appropriate storage pool, so that when we allocate space for the element we also reserve space for its reference counter.
    See the "Larger than it looks" blog post for more information on how this storage pool is implemented.

This second approach has now been implemented in the GNAT Components Collection, as the GNATCOLL.Refcount.Shared_Pointers package (the name is inspired by the C++ shared_ptr template, which plays a similar role).

Reviewing the Get API

The implementation described in Gem #97 was unsafe. Get would return an access to the element, which has several drawbacks (highlighted in a later Gem #107):

  • Users might try to free the element access
    This would of course result in invalid memory access later on, but even if users do not try this, that still makes the API more confusing because we need explicit comments to describe who has the ownership of the data.
  • Users might store the element access in a local variable or record
    This access might be freed at any time if the last reference to it disappears, and therefore accessing it is dangerous. Much better to always access it via the shared pointer.

Instead, the new API in GNATCOLL.Refcount.Shared_Pointers now returns a reference type for Get. Such types were introduced in Ada 2012 for the standard containers. They are basically records with access discriminants, and they use the Implicit_Dereference aspect. Here is an example:

    type Rec is tagged record
       Field : Integer;
    end record;
    type Rec_Access is access Rec;
    procedure Primitive1 (Self : Rec) is null;
    procedure Primitive2 (Self : access Rec) is null;

    type Reference_Type (E : access Rec'Class) is limited null record
       with Implicit_Dereference => E;

    R : Reference_Type := ...;   --  similar to using a shared pointer

    R.Field := 1;    --  No need for .all
    R.Primitive1;    --  No need for .all either
    R.E.Primitive2;  --  Here we need to use the discriminant

    P : access Rec := R.E;   --  valid, can't be deallocated
    P2 : Rec_Access := Rec_Access (R.E);  --   INVALID

Since the discriminant can only be read and not converted, it cannot easily be deallocated in practice.

The reference type itself has been made limited, so that it cannot be stored in a record. This encourages users to store the shared pointer instead, which in turn guarantees that the element remains live while the record exists.

We still do not prevent storing the access type in a record, but this has to be done via an anonymous access type. This could still result in invalid memory accesses though, as in the following example:


with GNATCOLL.Refcount;

package Support is
   type Rec is record
      Field : Integer;
   end record;

   package Pointers is new GNATCOLL.Refcount.Shared_Pointers (Rec);

   type Rec2 is record
      Field2 : access Rec;
   end record;
end Support;


with Ada.Text_IO; use Ada.Text_IO;
with Support; use Support;
procedure Main is
   R2 : Rec2;
begin
   declare
      R : Pointers.Ref;
   begin
      R.Set (Rec'(Field => 1));
      R2.Field2 := R.Get.Element;
   end;

   --  R has been finalized and R2.Field2 is now pointing
   --  to deallocated memory
   Put_Line (R2.Field2.Field'Img);   --  INVALID
end Main;

The GNAT compiler team recently significantly improved the performance for reference types. Their size is known at compile time, but since they are unconstrained types anyway, the compiler used to return them on the secondary stack, requiring extra calls to malloc. They are now just as efficient as using pointers, except that they are safer.

Reviewing the Set API

The intrusive version of the smart pointers described in Gem #97 had two versions of Set:

    procedure Set (Self : in out Ref; Data : Encapsulated'Class);
    procedure Set (Self : in out Ref; Data : access Encapsulated'Class);

This API is safe in the context of intrusive smart pointers, since they contain their own reference count. As a result, if we are to do something like:

    R1, R2 : My_Smart_Pointers.Ref;
    E : access Encapsulated'Class := new My_Element_Type;

    R1.Set (E);
    R2.Set (E);   --  twice

In this case, E will not be freed while either R1 or R2 exists.

This approach is however not suitable in the case of nonintrusive shared pointers, since they need to be allocated through the special storage pool we mentioned before. For that reason, only the first version of Set is provided, and will allocate the memory and copy the Data as needed.

However, there are cases where we need one extra subprogram. Let's say our Element_Type is a tagged type, with a primitive operation that uses an access parameter, as in:

    type Element_Type is tagged private;
    package Pointer is new Shared_Pointers (Element_Type);

    procedure Primitive (Self : access Element_Type) is
       R : Pointers.Ref;
    begin
       --  ??? How do we set R from Self
    end Primitive;

    R2 : Pointers.Ref := ...;
    R2.Get.Element.Primitive;

In the procedure, we definitely do not want to initialize R with a call like R.Set (Self.all). This would store a copy of Self in R, which likely is not what we want. This could even be dangerous since R will be finalized when Primitive exits, resulting in a freeing of the element stored in R (a copy of Self), which might end up freeing some data stored in Self. R2 itself since accesses Self, we might end up using deallocated memory.

Instead, the proper approach is to use:

     R.From_Element (Self);

This call must only be used when Self is indeed already managed by a shared pointer. It ensures that Self and its data will not be freed while either R or R2 exist.

Conclusion

The new package GNATCOLL.Refcount.Shared_Pointers provides a number of enhancements over the older GNATCOLL.Refcount.Smart_Pointers. The API is more flexible since any Element_Type can be put under control of a shared pointer. They are also more efficient thanks to various internal optimizations.

More importantly, perhaps, they are actually safer since the API will prevent cases where users might end up using deallocated memory.

This post did not mention other aspects which are of course preserved in this new API: task-safety and weak pointers.

]]>
Larger than it looks (storage pools) http://blog.adacore.com/header-storage-pools Wed, 13 May 2015 14:01:00 +0000 Emmanuel Briot http://blog.adacore.com/header-storage-pools

What are storage pools ?

Storage pools are Ada's way of letting users control memory management in their applications. A pool is an object associated with one or more specific types via a Storage_Pool attribute. Whenever the application performs memory allocation via the new operator, or frees memory via an instance of Unchecked_Deallocation, the storage pool Allocate and Deallocate primitives are used, instead of doing a direct call to the system's libraries for memory management.

There are multiple reasons why such a pool might be useful in your application. The typical two benefits are:

  • performance
    By using your own pool, you can allocate for instance a single large buffer and then reuse it for multiple objects in your application, without having to go call the slower system malloc every time.
  • automatic memory reclamation
    With the above approach, the buffer can be freed all at once, without freeing each of the objects individually.

Header storage pools

In this post, we will study another usage for storage pools. The goal is to allocate an extra header whenever we allocate memory for an object. This header is user-defined, and can be used to store per-object information, independently of the exact type of the object.

Here are two examples of use:

  • reference counting
    To implement reference counting for types, we need to store a counter for the number of references to the object. This counter can be stored in an extra header, rather than be allocated elsewhere in memory.
  • more efficient list nodes
    A doubly-linked list needs to store Previous and Next pointers for each of its nodes. We can store them in the extra header.

In both cases, storing information in the header results in improved performance, since we can do a single call to the system malloc, instead of two. This is further improved by taking advantage of hardware prefetching and caching, since the data from the extra header is kept close to the element data in memory.

Implementation details

Implementing such a pool is a bit tricky, since there are several details to take care of:

  • alignment
    A lot of processors expect memory blocks to be aligned on specific memory boundaries (bytes, words or double-words) for more efficient access. The system malloc takes care of that, of course, but that only means that our header will be properly aligned. We need some extra care to ensure that the element itself is also properly aligned.
  • fat pointers
    In Ada, some types require extra information. For instance, an unconstrained array needs access to its bounds (First and Last). GNAT typically uses a "fat pointer" for this purpose: the access itself is in fact a record of two pointers, one of which points to the bounds, the other points to the data. This representation is not appropriate in the case of the header storage pool, so we need to change the memory layout here.
  • efficiency
    Of course, we would like our storage pool to have a minimal overhead.

Let's start by setting up our storage pool. This will be a generic package, since we want the contents of the extra header to be user-defined. Since the handling of fat pointers is also type specific, we will pass the type of element as another formal parameter. This means that for each type, we will need a different storage pool, they can't be shared by multiple types.

In practice, the implementation we have now added in the GNAT Components Library (GNATCOLL.Storage_Pools.Headers) uses multiple generic packages to share code as much as possible and reduce the impact on code size, but for the sake of this presentation we will take a simpler approach.

 pragma Ada_2012;
 with System; use System;
 with System.Storage_Pools; use System.Storage_Pools;
 with System.Storage_Elements; use System.Storage_Elements;
​
 generic
    type Extra_Header is private;
    type Element_Type (<>) is limited private;
 package Headers is
    type Header_Pool is new Root_Storage_Pools with null record;
    overriding procedure Allocate
       (Self : in out Header_Pool;
        Addr : out System.Address;
        Size : Storage_Count;
        Alignment : Storage_Count);
    overriding procedure Deallocate
       (Self : in out Header_Pool;
        Addr : System.Address;
        Size : Storage_Count;
        Alignment : Storage_Count);
    overriding function Storage_Size
       (Self : Header_Pool) return Storage_Count
       is (Storage_Count'Last)
       with Inline => True;
 end Headers;

The extra header must be a constrained type, since we need to know its size in advance. The element type, however, can be any type (hence the use of keyword limited and the (<>) in its declaration) since its actual size will be known when the user calls the new operator (and that size is passed automatically to Allocate).

This example also uses Ada 2012, for instance to use the Inline aspect. This can easily be replaced with a pragma Inline, though. This aspect is used in a number of places, to satisfy our efficiency requirement.

Pointers to the element type

As we mentioned before, we need to ensure that the bounds for unconstrained arrays are stored next to the element, not in a separate memory block, to improve performance. This is done by setting the Size attribute on the type. When we set this size to that of a standard pointer, GNAT automatically changes the layout, so that we now have:

 +-------+------+---------+
 | First | Last | Element |
 +-------+------+---------+

The result of the new operator still returns a pointer to the Element in the above, so the application does not have direct access to First and Last (except, of course, via the 'First and 'Last attributes).

At the same time that we declare this access to the element, we also associate it with the pool so that allocations and deallocations are done by the pool:

-- within the generic package
Pool : Header_Pool;
type Element_Access is access all Element_Type;
for Element_Access'Size use Standard'Address_Size;
for Element_Access'Storage_Pool use Pool;

Size computation

Whenever Allocate is called (via the new operator), it is given the size we need to allocate for the element itself. This size does not include the bounds (First and Last in the previous section), nor the size of the extra header itself. So we need to do a bit of computation to get the exact size we need to request from the system.

The size for the bounds is given by the attribute Descriptor_Size. For most types, the value of this attribute is 0. However, in the case of unconstrained array types, it is the size of two integers.

In some cases, GNAT needs to allocate even more memory in fact. When a type is potentially a controlled type (a class-wide type for instance, or an array of controlled types), GNAT needs to store this type in a linked list so that it can find all such instances and properly finalize them when the program terminates. This extra size is the size of two pointers (Previous and Next) that are used to implement the list.

Unfortunately, there is currently no attribute that provides this size for a given element type. This is being added to the compiler, but for now we will need to systematically allocate this extra size, even if the type doesn't actually need those.

As a summary, the actual layout of the memory block we will allocate is the following:

 +--------+-------+------+----------+------+---------+
 | Header | First | Last | Previous | Next | Element |
 +--------+-------+------+----------+------+---------+

Note that most of the components (First, Last, Previous and Next) might not be needed (or even allocated) for all types of elements, so we are not wasting all that memory when it isn't needed.

There is still one more difficulty. The Element part in the above needs to be properly aligned on memory boundaries. So the size of the header needs to be rounded up.

The exact computation thus becomes:

     Finalization_Master_Size : constant Storage_Count :=
         2 * Standard'Address_Size;   --  for Previous and Next
     Extra_Offset : constant Storage_Count :=
         (Element_Type'Descriptor_Size + Finalization_Master_Size)
         / Storage_Unit;       --  convert from bits to bytes

     --  Compute the size for the header, rounded up for proper
     --  alignment
     Min_Header_Size_Bytes : constant Storage_Count :=
         Extra_Header'Size / Storage_Unit;
     Pointer_Size_Bytes : constant Storage_Count :=
         Standard'Address_Size / Storage_Unit;
     Header_Allocation : constant Storage_Count :=
         (Min_Header_Size_Bytes + Pointer_Size_Bytes - 1) / Pointer_Size_Bytes
         * Pointer_Size_Bytes;

And now we can implement our Allocate as such:

      with System.Memory;    use System.Memory;

      overriding procedure Allocate
         (Self      : in out Header_Pool;
          Addr      : out System.Address;
          Size      : System.Storage_Elements.Storage_Count;
          Alignment : System.Storage_Elements.Storage_Count)
      is
         pragma Unreferenced (Alignment);
         Aligned_Size : constant Storage_Count :=   --  bytes
            Size + Header_Allocation + Extra_Offset;

         --  Allocate via the system malloc
         Allocated : constant System.Address :=
            System.Memory.Alloc (size_t (Aligned_Size));
      begin
         --  Return a pointer to the "Element" part of the block
         Addr := To_Address
            (To_Integer (Allocated) +
            Integer_Address (Header_Allocation + Extra_Offset));
      end Allocate;

The last part to implement is the deallocation. This procedure gets passed the address returned by Allocate (so a pointer to the Element part of the block). We need to convert this to the address of the beginning of the block allocated by malloc, so that we can call free on it.

This computation is actually also needed to retrieve a pointer to the header so that the application can store and access the header data. So we will make a separate function for that.

      function Convert is new Ada.Unchecked_Conversion
         (System.Address, Extra_Header_Access);

      function Header_Of
         (Self : Header_Pool; Addr : System.Address)
         return access Extra_Header
         with Inline => True
      is
      begin
         return Convert (Addr - Header_Allocation - Extra_Offset);
      end Header_Of;

The implementation of Deallocate becomes the trivial

      function Convert is new Ada.Unchecked_Conversion
         (Extra_Header_Access, System.Address);

      overriding procedure Deallocate
         (Self      : in out Header_Pool;
          Addr      : System.Address;
          Size      : System.Storage_Elements.Storage_Count;
          Alignment : System.Storage_Elements.Storage_Count)
      is
         pragma Unreferenced (Alignment, Size);
         type Extra_Header_Access is access all Extra_Header;
         Header : constant Extra_Header_Access :=
            Extra_Header_Access (Header_Of (Self, Addr));
      begin
         System.Memory.Free (Convert (Header));
      end Deallocate;

Conclusion

As shown by this lengthy post, implementing storage pools is not a trivial task, as there are a lot of details to take into account. Since this is a very low-level part of the code, there are also often some compiler-specific things to take into account, like the layout of types in memory.

However, there are generally only a few storage pool types used in an application, so they should be made generic when possible, and the implementation reused as much as possible.

The implementation described above is now part of the GNAT Components Collection (GNATCOLL.Storage_Pools.Headers), in a slightly different way. We use two generic packages, so that most of the code can be shared among the pools that want the same extra_header, and only a few additions in a second, type-specific, package.

Our current implementation wastes some memory for the Previous and Next pointers used for potentially controlled types. An improvement in the compiler is forthcoming, so that we do not need to allocate space for these pointers if they are not needed by the compiler. This will be transparent for the application code, although of course it will save a bit of memory at run time.

Future posts will describe how this pool was reused to implement reference counting in an efficient manner.

]]>
What's in the Box? http://blog.adacore.com/whats-in-the-box Mon, 11 May 2015 17:44:46 +0000 Karen Mason http://blog.adacore.com/whats-in-the-box
Albert Lee - AdaCore IT Manager in the machine room

Forget about the black box - we want you to see what's in the AdaCore box!

Our machine room, the life blood of the company, is centered in the office space in a glass box and is pure eye candy for geeks and cable aficionados.

The machine room is 204 sq ft (19 sq meters), contains 73 machines (running the gamut from VxWorks, Linux, Windows, Android and more) 1.5 miles of cables and a state of the art fire suppression mechanism! Oh and the modern air intake system gives it a green energy seal of approval.

In the unlikely event of a fire; the smoke detectors allow 30 seconds before dispersing an agent called FM200 which will suppress the fire but not damage the equipment.

The economizer takes advantage of winters' freeze and draws cool air from the outside to cool the machine room and the heated air is then spilled to common office areas thus reducing heating costs.

We're proud to show-off the hardware side of our software!

]]>
Project P Open Workshop http://blog.adacore.com/project-p-open-workshop Wed, 06 May 2015 04:00:00 +0000 AdaCore Admin http://blog.adacore.com/project-p-open-workshop

Project-P Open Workshop

Model-driven engineering and qualifiable code generator

http://www.open-do.org/projects/p/

Project P is a three-year research project funded within the French FUI 2011 funding framework. It sees the collaboration of major industrial partners (Airbus, Astrium, Continental, Rockwell Collins, Safran, Thales), SMEs (AdaCore, Altair, Scilab Enterprise, STI), service companies (ACG, Aboard Engineering, Atos Origins) and research centers (CNRS, ENPC, INRIA, ONERA).

With the project now drawing to an end comes the final presentation of all the work that has been produced over the past 3 years...

Continental France, Project P leader, and all the project partners are pleased to invite you to the presentation of the final results of Project P

The event will take place on Tuesday 16th June 2015 from 1:30pm to 5:30pm at Pôle Systematic - Palaiseau, Amphitheatre building N°3 - Nano-Innov

To find out more about access, please click here: http://www.systematic-paris-region.org/sites/defau...

Agenda

1:45pm - 2:00pm: Welcome

2:00pm - 4:00pm : Presentations

Introduction to Project-P (Continental)

Development and qualification of a generic code generation framework (ACGs)

QGen: a qualifiable code generator for Simulink/Stateflow models (AdaCore)

Operational use of a code generator (Sagem and Thales Alenia Space)

Overview of other generators used in the project (AdaCore)

A code generator for UML models (ATOS Origin)

A formal verification perspective (IRIT)

4:00pm - 5:30pm : Workshop Demonstrations

Model Verification and Code Generation with QGen (AdaCore)

Hardware code generation with GAUT (LabSticc)

Software code generation for XCOS models (Scilab Enterprise)

Code generation from NSP language and Scicos (Inria/Altair/ENPC)

Multi-formalism (UML and Simulink) code generation (ATOS/AdaCore)

Generated code interoperability and schedulability analysis with SynDEx (INRIA)

Verification of formal requirements for generated code (IRIT, ONERA)

Description of formal requirements for generated code (IRIT)

Registration is mandatory and closes on June 10th 2015 …to do so, please click here

Philippe CUENOT
Continental Automotive France
Tel : 05-61-19-88-88
philippe.cuenot@continental-corporation.com

]]>
Verification on Ada code with Static and Dynamic Code Analysis - Webinar http://blog.adacore.com/verification-on-ada-code-with-static-and-dynamic-code-analysis-webinar Mon, 04 May 2015 12:20:57 +0000 Jamie Ayre http://blog.adacore.com/verification-on-ada-code-with-static-and-dynamic-code-analysis-webinar

One of the main challenges to get certification in Ada projects is the achievement of 100% code coverage but in most projects an amount of more than 95% structural coverage is hard to achieve. What can you do with the last 5% of code that can't be covered? DO-178C for example, provides a framework for the integration of various techniques in the development process to solve the problem. In this webinar you learn how static analysis and dynamic testing can help complete analysis for pieces of code that are not covered.


This hour-long webinar takes place Thursday May 7, 2015 at 10:00am UK, 11:00am CET. To register, please click here.

]]>
The Year for #AdaLove http://blog.adacore.com/adalove Mon, 27 Apr 2015 19:09:00 +0000 Karen Mason http://blog.adacore.com/adalove

Despite her famously sharp analytical mind, it’s unlikely Ada Lovelace could have predicted the durability of her legacy as the world’s first computer programmer and pioneer for women in computing.

In 2015, as we celebrate the 200th anniversary of Ada’s birth on December 10, it’s a great time to reflect upon, acknowledge, promote and commend the contributions of women in science, technology, engineering and mathematics (STEM) disciplines.

Ada, sometimes called the Enchantress of Numbers, was born Augusta Ada Byron and the daughter of famed British poet Lord Byron. Although Byron was not around for her formative years, his influence on her life carried into her work, leading her to proclaim herself a “poetical scientist” who understood the importance of imagination in math and science.

Ada’s most influential work was with Charles Babbage, creator of the Analytical Engine, which is considered the world’s first computer. Ada was Babbage’s voice to the world, and it was her critical combination of science and poetry that allowed Ada to recognize the value of Babbage’s plan and predict likely outcomes based on his ideas.

A programming language was named in her honor, and today the name Ada Lovelace is strongly associated with computer programming. The Ada language, which was designed for large, long-lived applications where safety, security, and reliability are critical, is a fitting tribute to her early work in computing.

Please join us and use the hashtag, #AdaLove and tweet us @AdaCoreCompany all year long sharing your photos and memories of your journey into programming or other STEM disciplines. Please also share stories and photos of women who inspired you.We can’t wait to read your contributions and spread the #AdaLove.

For more information about Ada’s life and career, an excellent resource is Betty Toole’s Ada: Enchantress of Numbers: Poetical Science.



]]>
French Intelligence Bill: A Minority Report http://blog.adacore.com/new-french-intelligence-bill-a-minority-report Thu, 16 Apr 2015 04:00:00 +0000 Romain Berrendonner http://blog.adacore.com/new-french-intelligence-bill-a-minority-report

Following the terrorist attacks that occurred in Paris earlier this year, the French government is proposing a bill that will, supposedly, improve the control over intelligence services and give legal ground to some of the intelligence-gathering methods these services are already known to use, ie. render legal, the previously illegal, Internet wiretapping. Interestingly, opponents of the this time bill include not only the usual civil liberty activists such as La Quadrature du Net, Amnesty International or La ligue des droits de l’Homme. Professional trade unions, such as Syntec, companies, such as OVH and even official bodies, such as the CNIL and the Défenseur des droits also expressed an unusually vocal opposition to the proposed bill.

In this blog post, I’d like to make a short presentation of the proposed bill and explain why and how AdaCore would also be affected if it became law in its current format, focusing on three main aspects of the bill : the legal oversight of intelligence services; their scope of action; the mass collection of data permitted by the bill.

This is not a full legal analysis of the bill, for which other sources are available. I will leave also some interesting questions aside such as the motivation for a new anti-terror bill only five months after a previous one was introduced, or the reason for focusing on Internet communications in the wake of January attacks whose perpetrators radicalized in prison, not on the Internet.

An insufficient legal oversight

According to the bill, the use of wiretapping techniques can be ordered only by the Prime minister, after a body named the National Commission of Intelligence Techniques Control provides their opinion. The Prime Minister, however, is not bound by this opinion of the commission and, incases of emergency, does not even have to wait for its opinion.

This new independent administrative body is made of nine members, and includes two representatives, two senators, as well as two judicial judges, two administrative judges and one technical expert. It is similar in spirit to other similar bodies, such as the CNIL, the personal data watchdog created by the 1978 law which inspired directive 95/46/EC. However, its powers are considerably weaker : it can only emit a “recommendation” to the Prime Minister (art. L 821-1 and following) in case of infringement and submit the case to the Conseil d'Etat, the supreme court of the administrative order, if the Prime Minister disregard their recommendation. It has no repressive power of its own.

The bill also creates a special procedure before the Conseil d’Etat opened to the commission and to persons who complain about wiretapping. However, if the Conseil d’Etat can establish that the legal conditions for wiretapping are met based on the documents made available by the Commission, the plaintiff has no access to these classified documents (art. L773-1 and following of the Administrative Justice Code as modified by the bill). If the wiretapping is illegal, the Conseil d’Etat can enjoin the Prime Minister to stop it and destroy the records.

Many opponents complain that there is no judicial judge involved in this process, but this is not the most significant issue at stake here I believe, as the Conseil d’Etat, despite its name, has a long-standing tradition of independence from the government and legal rigor.

What is really significant here is that plaintiff would not have access to the case documents, a huge breach to the due process of law which will remove any possibility to effectively litigate : to all practical extents, there will be no effective access to the courts for those who are subject to wiretapping.

A broad scope of action

In this context, defining the scope of action of intelligence services is even more important to be able to have a minimum level of control. But the bill, beyond the expected “national security”, “preventing terrorism”, and “preventing organized crime” also covers “prevention of collective violence susceptible to gravely alter public peace” and protection of “the economical and scientific interests of France” (art. L811-3).

What does that mean for a company like AdaCore? Most of our customers are large companies, or subcontractors of large companies, dealing with the aerospace, defense, and railway domain. Typically the kind of markets where mega requests to tender are made, and fierce competition, supported by national interests, take place : just keep in mind what happened during the Saudi 1994 airliner tender. With this bill, targeting by the French services is acknowledged and documented and may deter American, Russian, or Indian customers to trust AdaCore. This is not pure theory : some of our large European customers have become much more sensitive to the issue of wiretapping in the wake of Edward Snowden’s revelations and we have had to specifically address their concerns.

In terms of geographic area, the jurisdiction of the commission is limited to the French territory. Not only does the proposed bill not put in place any control for wiretapping whenever one of the end points is located abroad (art. L854-1), but it also explicitly states that special agents are not criminally liable for wiretapping undertaken abroad (art. 323-8 of the penal code, as modified by the bill). Given how distributed the Internet is, this is probably an open door to the subcontracting of intrusive wiretapping activities to third party states.

A mass gathering of data

Another much discussed issue raised by the bill is the possibility offered to intelligence services to order ISP and providers of various Internet Services to “detect, by means of an automated process, any suspicious sequence of connection data, whose anonymity will be lifted only if a terror threat is revealed” (exposé des motifs and art. L 851-4). What is described with this falsely legalistic and truly fuzzy formula is the massive harvest of connection meta data by ISP and service providers on behalf of the government, with the fundamental inconsistency, clearly pointed out by the CNIL, that this supposedly anonymous collection of data can be turned to non-anonymous data on demand.

In theory, this possibility to gather huge amounts of individual data is restricted to “the prevention of terrorism”. However, it is hard to see how this will be effectively thus restricted. In particular, it is well-known that terrorists need to fund their activities and therefore commit ordinary crimes such as money laundering. So it is more than likely that Intelligence Services will use this broadly rather than narrowly, and target the business world like any other.

In addition, I must say, as a computer scientist and legal graduate, the fact there are people that believe it is possible from Internet meta data to infer even presumably that someone is planning a terror attack, is much more reminiscent of Minority Report than of rational crime fighting.In the movie at least, the Precogs were human beings ...

Conclusion

To summarize, for AdaCore, this bill means that our foreign customers can legally be targeted by French intelligence services; that these services can massively harvest internet metadata from our business communications; and that we have no legal course of action to stop this.

I trully hope that, in order to keep the same level of trust with our customers, we will not have to reorganize our services so that customers from a specific region are served only by AdaCore staff based in this region. AdaCore leverages on rare technical expertise located both in the United States and in Europe, and only this unique mix allows us to provide the complex services our customers appreciate.

Over and beyond our own situation, we can only refuse a society where all companies providing Internet services would be mandated to act as police auxiliary; where the end points of each communication would be identified, and revealed to intelligence services; in one word, a society where each citizen and business would be treated as a potential suspect.

Unless explicitly said otherwise, all references to articles below refer to articles of the homeland security code (Code de la sécurité intérieure), as they would be modified by the bill. Also, this blog post does not take into account amendments made to the original bill by the National Assembly

]]>
The latest Mixed Programming with Ada lectures at the AdaCore University http://blog.adacore.com/the-latest-mixed-programming-with-ada-lectures-at-the-adacore-university Fri, 27 Mar 2015 08:46:00 +0000 Martyn Pike http://blog.adacore.com/the-latest-mixed-programming-with-ada-lectures-at-the-adacore-university

I recently joined AdaCore as a Technical Account Manager with an initial focus on the UK and Scandinavian regions, but for the last 12 months I've been busy working on the AdaCore University. The most recent addition to which is a course on Mixed Language Programming with Ada, and it includes lectures on the integration of Ada with C, C++ and Java. The course covers some advanced topics like mixed language object orientation, techniques for using Ada strong typing to combat known issue with C pointers and the pitfalls that are encountered when mixing native Ada code with Java at runtime. This course clearly demonstrates that Ada has strong support for integration with C, C++ and Java and it proves there are no technical barriers to its adoption in modern mixed language software systems.

]]>
A Building Code for Building Code http://blog.adacore.com/a-building-code-for-building-code Wed, 25 Mar 2015 04:00:00 +0000 Yannick Moy http://blog.adacore.com/a-building-code-for-building-code

If you can't make sense of the title of this post, you may need to read the recent article about it in Communications of the ACM. In this article, Carl Landwehr, a renowned scientific expert on security, defends the view that the software engineering community is doing overall a poor job at securing our global information system:

To a disturbing extent, however, the kinds of underlying flaws exploited by attackers have not changed very much. [...] One of the most widespread vulnerabilities found recently, the so-called Heartbleed flaw in OpenSSL, was [...] failure to apply adequate bounds-checking to a memory buffer.

and that this is mostly avoidable by putting what we know works to work:

There has been substantial progress in the past 20 years in the techniques of static and dynamic analysis of software, both at the programming language level and at the level of binary analysis. [...] It would be feasible for a building code to require evidence that software for systems of particular concern (for example, for self-driving cars or SCADA systems) is free of the kinds of vulnerabilities that can be detected automatically in this fashion.

to the point that most vulnerabilities could be completely avoided by design if we cared enough:

Indeed, through judicious choice of programming languages and frameworks, many kinds of vulnerabilities can be eliminated entirely. Evidence that a specified set of languages and tools had indeed been used to produce the finished product would need to be evaluated.

Shocking! Or so it should appear. But the reality is that we are now used to not being able to rely on software in our everyday lives. We're more than 10 years past the "Trustworthy Computing" initiative by Microsoft to counter the effect caused by the viruses Code Red and Nimda in 2001. We've had Stuxnet and Heartbleed since then, so we know we're not safe yet.

But Carl Landwehr is not just shooting in the dark. This article in CACM is the latest step in a coordinated effort to raise the awareness of the software engineering community on these matters. It started with an essay in 2013 in which he describes in more details his point that today we're building cathedrals of sand. The following sentence neatly summarizes what we're advocating as developers of the Ada and SPARK programming environments:

At least use development practices that minimize, and, where possible, rule out these kinds of flaws. Use programming languages that make buffer overflows infeasible. Use static and dynamic analysis techniques on software for which source code is available to confirm the absence of whole classes of flaws. The software won’t be perfect but it will be measurably better.

Of course, web development and the development of critical embedded software is not alike. Different domains will require different sets of programming languages and techniques. This is why Carl Landwehr organized a workshop last year to develop a Building Code for medical device software. The results of which were published in this report. Of particular interest in relation to the choice of programming language and analysis techniques is section B of that report on "Elements intended to avoid / detect / remove specific types of vulnerabilities at the implementation stage":

  1. Use of Memory-Safe Languages
  2. Language Subsetting
  3. Automated Memory Safety Error Mitigation and Compiler-Enforced Buffer Overflow Elimination
  4. Use of Secure Coding Standards
  5. Automated Thread Safety Analysis
  6. Automated Analysis of Programs (Source/Binary) for Critical Properties
  7. Accredited Cryptographic Algorithms and Implementation
  8. Modified Condition Decision Coverage
  9. Operational Use Case Identification and Removal of Unused Functions

Ada and SPARK are ideally suited to address items 1 to 6, while items 7 to 9 are largely language independent (and thus apply equally well to Ada and SPARK as to other languages). This is no accident as these goals were explicit design goals for both languages. They do not need to be retrofitted after the fact in an ad-hoc and partial way. Funnily, the analogy between building construction and software construction on which Carl Landwehr 'builds' was already used in 2003 by Franco Gasperoni in slides for teaching Ada, and the footnote says it originally comes from a presentation from 1999 by Tucker Taft (Franco and Tuck are both members of AdaCore's executive team), just check these two slides:

construction analogy
tool and material quality

The initial focus of Carl Landwehr on the medical device industry echoes the presentation from Professor Harold Thimbleby at the HIS conference last year, where he suggested the use of labels for software quality similar to the labels we have in Europe for energy consumption of home appliances. He even showed us a label he had made online, explaining that an interesting side effect of this regulatory constraint has been the need to update the initial scale (of A to G) with additional labels A+, A++ and A+++ to keep up with the advances in energy efficiency made by the industry. Indeed, who would like to rank B in a scale that starts at A+++? If I made my own B label for a drone collision avoidance software, would you be interested in buying it?

]]>
QGen on Embedded News TV http://blog.adacore.com/embedded-news-tv-qgen Thu, 12 Mar 2015 15:36:00 +0000 Jamie Ayre http://blog.adacore.com/embedded-news-tv-qgen

Embedded News TV caught up with our own Matteo Bordin to talk about QGen. Matteo provides a nice overview of QGen and it's position in the industry as the need for safe and secure software becomes increasingly important.

]]>
20 years on... http://blog.adacore.com/20-years-on Wed, 11 Mar 2015 04:00:00 +0000 Jamie Ayre http://blog.adacore.com/20-years-on

20 Years of AdaCore: Company as Committed as Ever on Safety-Critical Software Solutions

Twenty years ago – and with just one paid employee – AdaCore was born following a 16-year gestation at New York University. Dr. Robert Dewar, one of our co-founders and Emeritus Professor of Computer Science at NYU, initiated and ran an Ada-focused research program at the university after getting his first taste of the Ada language (the namesake of Ada Lovelace, the world's first female programmer). When his NYU team was tasked by the Ada 9X Project Office (Department of Defense) with making a freely available Ada 9X compiler, the GNAT project officially began.

By 1994, GNAT was ready for commercial exploitation, but that couldn't happen without a company behind it. Ada Core Technologies was formed, founded by Dr. Dewar, Ed Schonberg and Richard Kenner, and hired Gary Dismukes as its first employee. We launched with a contract in hand from Silicon Graphics, Inc. (SGI). The following year, the ISO published the Ada 95 standard and AdaCore's signature product, GNAT Pro, became commercially available. Other early AdaCore clients included General Dynamics, Mitsubishi, Raytheon and the United States Army.

Today, the world and AdaCore look quite different from two decades back. Our team and our business have grown substantially from those early years. Today, we have customers around the world and have global headquarters in New York and Paris. AdaCore customers are building space-based systems, commercial aircraft, military systems, air traffic management and control systems, railway systems, medical devices, and financial services systems using our solutions.

Over the years, AdaCore has developed a variety of breakthrough products including CodePeer (developed through a partnership with our customer SofCheck) to support Ada 2005 and Ada 2012, as well as SPARK Pro 8.1.0 while adding additional multinational customers like Thales to its portfolio. In 2004, we created the GNAT Academic Program (GAP), which encourages the use of the Ada language in college curriculum. To date, hundreds of colleges are participating in the program. AdaCore also entered into strategic partnerships with Wind River, SofCheck (with whom we later merged) and Praxis Critical Systems (now Altran); and with a growing user base, AdaCore launched its first GNAT Pro User Day in Paris in 2011. We are also working hard to provide online training for the safety-critical software developers of the future through AdaCore University (u.adacore.com). To date we have four courses available and will be adding more through the coming years.

Through it all, we've not strayed from our roots. Safety- and security-critical software solutions built around Ada and the continued promotion and expansion of the Ada language always have been our focus. Along the way, we've built a long-lived company with a stellar team, top-notch customer service, with quality and reliability in everything we do. Our product line has grown to reflect the shift in the need for critical software development solutions to include three other flagship tools to accompany GNAT Pro: the CodePeer advanced static analysis tool, the SPARK Pro verification environment (based on formal methods), and the QGen qualifiable and customizable code generator.

With 20 years down, we're not resting on our accolades. The need for safety-critical software solutions is more pronounced than ever. We reminisced about our past at AdaCore's annual company meeting in Milan, but the real focus was kicking off our next 20 years.

]]>
AdaCore Releases GNAT Pro 7.3, QGen 1.0 and GNATdashboard 1.0 http://blog.adacore.com/adacore-releases-gnat-pro-7-3-and-qgen-1-0 Thu, 05 Mar 2015 05:00:00 +0000 Olivier Ramonat http://blog.adacore.com/adacore-releases-gnat-pro-7-3-and-qgen-1-0

February saw the annual customer release of a number of important products. This is no mean task when you consider the fact that GNAT Pro is available on over 50 platforms and supports over 150 runtime profiles (ranging from Full Ada Support to the very restricted Zero Footprint Profile suitable for safety-critical development). All in all, from the branching of the preview version to the customer release it takes us nearly 4 months to package everything up! Quality is assured through the internally developed AdaCore Factory.

Below are the details of the enhancements, many of which come directly from customer input:

GNAT Pro 7.3.1

GNAT Pro 7.3.1 has upgraded the backend (GCC 4.9) and debugging technologies (GDB 7.8) and incorporates over 175 new features. These include improved diagnostic messages and warning configuration, extended support for non default endianness, a math library on bare board platforms, support for large files on 32bits systems, improved support for inlining, overflow checks enabled by default, enhanced code generation and debugging capabilities.

In addition, most GNAT Pro tools now support aggregate projects. Parallel and incremental processing has been also added to many tools (e.g. gnat2xml, gnatmetric, ...), and support for stubbing units has been added in GNATtest.

CodePeer 3.0.1

- DO-178B and EN50128 qualification,
- support for IEEE 754 floating point semantics,
- enhanced project file support,
- improved support for other compilers,
- new and enhanced messages.

SPARK 15.0.1

- language features: support for tagged types and dynamic dispatching, support for proof-only ("ghost") code,
- proof capabilities: new automatic generation of contracts, parallel proofs, better handling of arrays, integers and floating point proofs, support for manual provers,
- improved handling and categorization of messages and assumptions.

GPS 6.1.1 & GNATbench 2.9.1

- improved source navigation (engine using a database),
- simplified workflow and project templates for bare board platforms,
- better Ada 2012 and SPARK 2014 support,
- support for Eclipse 4.4 Luna and Workbench 4.0.

This year's release also saw 2 new products launched:

QGen 1.0.1

QGen is a qualifiable and customizable code generator producing Ada (SPARK-compliant) and C (MISRA C-compliant) from Simulink and Stateflow models. It is particularly suited for usage in regulated industries by organizations who develop high-integrity real-time applications with Simulink/Stateflow and who want to decrease their V&V costs on generated source code.

GNATdashboard 1.0.1

A visual tool that supports quality assurance activities, integrating and aggregating the results of AdaCore's static and dynamic analysis tools within a common interface.


]]>
Testing, Static Analysis, and Formal Verification http://blog.adacore.com/testing-static-formal Fri, 20 Feb 2015 05:00:00 +0000 Johannes Kanig http://blog.adacore.com/testing-static-formal

I've recently written an article (in two parts) over at Electronic Design about applying different methods of verification to the same small piece of code. The code in question is an implementation of binary search, and I applied Testing, Static Analysis (using the AdaCore tool CodePeer) and Formal Verification (using the AdaCore tool SPARK 2014).

In the First Part, I started with one implementation of binary search, applied a quite naive testing strategy, which appeared to be relatively comprehensive, and managed to find one bug. I then applied the CodePeer tool, which found another problem. In the Second Part, I wrote Ada 2012 contracts and applied the SPARK 2014 tool to the code. I not only found two more problems, but also managed to prove that the code is entirely free of run-time errors and always provides the correct answer.

]]>
AdaCore at FOSDEM'15 http://blog.adacore.com/adacore-at-fosdem15 Tue, 10 Feb 2015 05:00:00 +0000 Tristan Gingold http://blog.adacore.com/adacore-at-fosdem15

I was at Bruxelles on January 31st to present the components of GNAT GPL 2015 : SPARK 2014 and GNAT GPL for ARM bare-board. This is not unrelated to a previous blog entry on Tetris in SPARK on ARM Cortex M4, in particular I presented that Tetris demo (I brought some boards with me and despite the simple package, none were broken!). The slides contain technical details on the ravenscar profile (main principles), how to build a program for the stm32f4-discovery board and how to port the runtime. There are also less technical slides such as why we choose the stm32f4 board and photos of some graphical demos. As that could be useful to anyone interested in Ravenscar or in porting the runtime to other boards or other platforms, we've made the slides available here.

Attachments

]]>
ProofInUse is coming! http://blog.adacore.com/proofinuse-is-coming Fri, 23 Jan 2015 05:00:00 +0000 Lena Comar http://blog.adacore.com/proofinuse-is-coming

For lovers of verification tools and critical system (we know you're out there!), we are very excited to present ProofInUse!

The common laboratory ProofInUse is a beautiful love story between the public research institutes (Inria) and private industry (AdaCore) whose objective is to develop verification tools based on mathematical proof for industrial users.

The laboratory will be launched on February 2nd 2015 at an open event (after registration) taking place at Novotel Paris Gare Montparnasse. During this day the two directors of the laboratory will talk about the scope of the project and the technology involved. Moreover at the end of the day there will be a Q&A session where researchers will mix with people from industry such as Dassault Aviation or Airbus Defence & Space.

You can discover all the details of this event at the link below. Be sure not to miss the cocktail at the end of the day ;) Following this 'kick-off' the lab will be located in AdaCore's Paris offices promoting interaction between our own experts and Inria's. A number of engineers from the project will be attending the launch and will be pleased to meet and discuss the project with you.

Please register for the event here : http://www.spark-2014.org/proofinuse/kickoff
For further information about the SPARK technology: http://www.spark-2014.org/proofinuse/

]]>
A Busy Schedule Ahead! http://blog.adacore.com/discover-adacores-upcoming-events Thu, 15 Jan 2015 05:00:00 +0000 AdaCore Admin http://blog.adacore.com/discover-adacores-upcoming-events

If you have a passion for Ada, need more information on our technology or would just like to have a chat, there are a couple of upcoming events where we'd love to meet up. What's more, we'll be launching our brand new product QGen at Embedded World!

Embedded World runs Feb 24th-26th and takes place at the Nuremberg Exhibition Centre in Germany. If you haven't already been, it is billed as a "leading international conference focusing exclusively on embedded technologies". It rarely disappoints and reflects trends in the sector with around 900 exhibitors and many thousands of visitors. AdaCore will be exhibiting in hall 4 stand 149.

Our second interesting event actually falls on the same date (Feb 24th-26th)! The Certification Together conference is held every two years in Toulouse, France. Presented as "a unique opportunity to meet a top-level panel of more than 300 international experts and specialists from aeronautical companies, Certification Authorities, experts and training companies, subcontractors, solution suppliers, engineering schools and R&D labs" it also very much worth a visit.

At both conference, AdaCore staff will be happy to present our new tool QGen, a qualifiable and customizable code generator producing Ada (SPARK-compliant) and C (MISRA C-compliant) from Simulink and Stateflow models. It is particularly suited for usage in regulated industries by organizations who develop high-integrity real-time applications with Simulink/Stateflow and who want to decrease their V&V costs on generated source code.

We look forward to seeing you at either one!

]]>
Tetris in SPARK on ARM Cortex M4 http://blog.adacore.com/tetris-in-spark-on-arm-cortex-m4 Wed, 07 Jan 2015 08:00:00 +0000 Tristan Gingold http://blog.adacore.com/tetris-in-spark-on-arm-cortex-m4

Tetris is a well-known game from the 80's, which has been ported in many versions to all game platforms since then. There are even versions of Tetris written in Ada. But there was no version of Tetris written in SPARK, so we've repaired that injustice. Also, there was no version of Tetris for the Atmel SAM4S ARM processor, another injustice we've repaired.

The truth is that our colleague Quentin Ochem was looking for a flashy demo for GNAT using SPARK on ARM, to run on the SAM4S Xplained Pro Evaluation Kit of our partner Atmel. Luckily, this kit has an extension with a small rectangular display that made us think immediately of Tetris. Add to that the 5 buttons overall between the main card and the extension, and we had all the necessary hardware.

Now, how do you make a Tetris in SPARK for ARM? First, the ingredients:

  • a SAM4S Xplained Pro Evaluation Kit + OLED1 Xplained Pro extension + Atmel manuals
  • GNAT GPL 2014 for ARM
  • a recent wavefront of SPARK Pro 15.1 (*)
  • the WikiPedia page describing Tetris rules
  • a webpage describing the Super Rotation System (SRS) for Tetris
  • an engineer who knows SPARK
  • an engineer who knows ARM

(*) If you don't have access to SPARK Pro 15.1, you can use SPARK GPL 2014, but expect some differences with the verification results presented here.

Count 2 days for designing, coding and proving the logic of the game in SPARK, another 2 days for developing the BSP for the board, and 0.5 day for putting it all together. Now the detailed instructions.

The whole sources can be downloaded in the tetris.tgz archive attached.

A Core Game Logic in SPARK

SPARK is a subset of Ada that can be analyzed very precisely for checking global data usage, data initialization, program integrity and functional correctness. Mostly, it excludes pointers and tasking, which is not a problem for our Tetris game.

We modeled the display (the board) as an array of Y_Size lines, where each line is an array of X_Size cells, and the origin is at the top left corner:

A cell is either empty, or occupied by a piece in I, O, J, L, S, T, Z (the name of the piece hints to what form it takes...), and the piece falling as a record with the following components:

  • a shape in I, O, J, L, S, T, Z
  • a direction (North, East, South or West) according to SRS rules
  • a pair of (X,Y) coordinates in the board, corresponding to the coordinate of the top-left cell of the square box enclosing the piece in SRS

Two global variables Cur_Board and Cur_Piece store the current value of the board and falling piece. Finally, SRS rules are encoded in Boolean matrices defining the masks for oriented shapes (where True stands for an occupied cell, and False for an empty cell).

All the above can be expressed straightforwardly in SPARK as follows:

--  possible content of the board cells
type Cell is (Empty, I, O, J, L, S, T, Z);

--  subset of cells that correspond to a shape
subtype Shape is Cell range I .. Z;

--  subset of shapes that fits in a 3 x 3 box, that is, all expect I and O
subtype Three_Shape is Cell range J .. Z;

--  the board is a matrix of X_Size x Y_Size cells, where the origin (1,1)
--  is at the top left corner

X_Size : constant := 10;
Y_Size : constant := 50;

subtype X_Coord is Integer range 1 .. X_Size;
subtype Y_Coord is Integer range 1 .. Y_Size;

type Line is array (X_Coord) of Cell;
type Board is array (Y_Coord) of Line;

Cur_Board : Board;

--  the current piece has a shape, a direction, and a coordinate for the
--  top left corner of the square box enclosing the piece:
--    a 2 x 2 box for shape O
--    a 3 x 3 box for all shapes except I and O
--    a 4 x 4 box for shape I

subtype PX_Coord is Integer range -1 .. X_Size - 1;
subtype PY_Coord is Integer range -1 .. Y_Size - 1;

type Direction is (North, East, South, West);

type Piece is record
   S : Shape;
   D : Direction;
   X : PX_Coord;
   Y : PY_Coord;
end record;

Cur_Piece : Piece;

--  orientations of shapes are taken from the Super Rotation System at
--  http://tetris.wikia.com/wiki/SRS
--    shape O has no orientation
--    shape I has 4 orientations, which all fit in the 4 x 4 box
--    shapes except I and O have 4 orientations, which all fit in the 3 x 3 box

--  Note that Possible_I_Shapes and Possible_Three_Shapes should be accessed
--  with Y component first, then X component, so that higher value for
--  Direction correspond indeed to clockwise movement.

subtype I_Delta is Integer range 0 .. 3;
type Oriented_I_Shape is array (I_Delta, I_Delta) of Boolean;
subtype Three_Delta is Integer range 0 .. 2;
type Oriented_Three_Shape is array (Three_Delta, Three_Delta) of Boolean;

--  orientations for I

Possible_I_Shapes : constant array (Direction) of Oriented_I_Shape :=
  (((False, False, False, False), (True,  True,  True,  True),  (False, False, False, False), (False, False, False, False)),
   ((False, False, True,  False), (False, False, True,  False), (False, False, True,  False), (False, False, True,  False)),
   ((False, False, False, False), (False, False, False, False), (True,  True,  True,  True),  (False, False, False, False)),
   ((False, True,  False, False), (False, True,  False, False), (False, True,  False, False), (False, True,  False, False)));

Possible_Three_Shapes : constant array (Three_Shape, Direction) of Oriented_Three_Shape :=
  (--  orientations for J
   (((True, False, False), (True,  True,  True),  (False, False, False)),
    ((False, True, True), (False,  True,  False),  (False, True, False)),
    ((False, False, False), (True,  True,  True),  (False, False, True)),
    ((False, True, False), (False,  True,  False),  (True, True, False))),

   --  orientations for L
   (((False, False, True), (True,  True,  True),  (False, False, False)),
    ((False, True, False), (False,  True,  False),  (False, True, True)),
    ((False, False, False), (True,  True,  True),  (True, False, False)),
    ((True, True, False), (False,  True,  False),  (False, True, False))),

   --  orientations for S
   (((False, True, True), (True,  True,  False),  (False, False, False)),
    ((False, True, False), (False,  True,  True),  (False, False, True)),
    ((False, False, False), (False,  True,  True),  (True, True, False)),
    ((True, False, False), (True,  True,  False),  (False, True, False))),

   --  orientations for T
   (((False, True, False), (True,  True,  True),  (False, False, False)),
    ((False, True, False), (False,  True,  True),  (False, True, False)),
    ((False, False, False), (True,  True,  True),  (False, True, False)),
    ((False, True, False), (True,  True,  False),  (False, True, False))),

   --  orientations for Z
   (((True, True, False), (False,  True,  True),  (False, False, False)),
    ((False, False, True), (False,  True,  True),  (False, True, False)),
    ((False, False, False), (True,  True,  False),  (False, True, True)),
    ((False, True, False), (True,  True,  False),  (True, False, False))));

Both the user (by punching buttons) and the main loop of the game (by moving the falling piece down), can perform one of 5 elementary actions, whose names are self explanatory: Move_Left, Move_Right, Move_Down, Turn_Counter_Clockwise, Turn_Clockwise. Dropping the piece is not an elementary action, as it can be obtained by rapidly moving the piece down.

The game logic provides the following API:

  • Do_Action: attempts an action and returns whether it was successfully applied or not
  • Include_Piece_In_Board: transition from state where a piece is falling to its integration in the board when it cannot fall anymore
  • Delete_Complete_Lines: remove all complete lines from the board

Note that all 3 services are implemented as procedures in SPARK, even Do_Action which could be implemented as a function in full Ada, because functions are not allowed to write to global variables in SPARK (that is, functions cannot perform side-effects in SPARK).

There are a few additional functions to return the new value of the falling piece after a move (Move), to know whether a line in the board is empty (Is_Empty_Line) and whether a piece occupies only empty cells of the board (No_Overlap), that is, it fits in the board and does not conflict with already occupied cells.

The complete game logic is only 165 sloc according to GNATmetrics (see tetris_initial.adx in the archive attached). The tool GNATprove in the SPARK toolset can check that this is valid SPARK by using switch –mode=check or equivalently menu SPARK➤Examine File in GPS. See files tetris_initial.ad? in the tetris_core.tgz archive attached.

Note that we use expression functions to implement most small query functions. This allows both to define these functions in a spec file, and to prepare for proof, as the body of these functions acts as an implicit postcondition.

Proving Absence of Run-Time Errors in Tetris Code

Of course, the benefit of writing the core game logic in SPARK is that we can now apply the SPARK analysis tools to demonstrate that the implementation is free from certain classes or errors, and that it complies with its specification.

The most immediate analysis is obtained by running GNATprove with switch –mode=flow or equivalently menu SPARK➤Examine File in GPS. Here, it returns without any message, which means that there are no possible reads of uninitialized data. Note that, as we did not provide data dependencies on subprograms (global variables that are input and output of subprograms), GNATprove generates them from the implementation.

The next step is to add data dependencies or flow dependencies on subprograms, so that GNATprove checks that the implementation of subprograms does not read other global variables than specified (which may not be initialized), does not write other global variables than specified, and derives output values from input values as specified. Here, we settled for specifying only data dependencies, as flow dependencies would not give more information (as all outputs depend on all inputs):

procedure Include_Piece_In_Board with
  Global => (Input => Cur_Piece, In_Out => Cur_Board);
--  transition from state where a piece is falling to its integration in the
--  board when it cannot fall anymore.

procedure Delete_Complete_Lines with
  Global => (In_Out => Cur_Board);
--  remove all complete lines from the board

Running again GNATprove in flow analysis mode, it returns without any message, which means that procedures Include_Piece_In_Board and Delete_Complete_Lines access global variables as specified. See files tetris_flow.ad? in the tetris_core.tgz archive attached.

The next step if to check whether the program may raise a run-time error. First, we need to specify with preconditions the correct calling context for subprograms. Here, we only add a precondition to Include_Piece_In_Board to state that the piece should not overlap with the board:

procedure Include_Piece_In_Board with
  Global => (Input => Cur_Piece, In_Out => Cur_Board),
  Pre    => No_Overlap (Cur_Board, Cur_Piece);

This time, we run GNATprove in proof mode, either with switch –mode=prove or equivalently menu SPARK➤Prove File in GPS. GNATprove returns with 7 messages: 3 possible array accesses out of bounds in procedure Include_Piece_In_Board, 3 possible violations of scalar variable range in function Move, and 1 similar violation in procedure Delete_Complete_Lines.

The message on Delete_Complete_Lines points to a possible range check failure when decrementing variable To_Line. This is a false alarm, as To_Line is decremented at most the number of times the loop is run, which is To_Line - 1. As usual when applying GNATprove to a subprogram with a loop, we must provide some information about the variables modified in the loop in the form of a loop invariant:

pragma Loop_Invariant (From_Line < To_Line);

With this loop invariant, GNATprove proves there is no possible range check failure.

Although the possible array index messages in Include_Piece_In_Board also occur inside loops, the situation is different here: the indexes used to access array Cur_Board are not modified in the loop, so no loop invariant is needed. Instead, the relevant part of the No_Overlap precondition that is needed to prove each case needs to be asserted as follows:

   when I =>
      --  intermediate assertion needed for proof
      pragma Assert
        (for all YY in I_Delta =>
           (for all XX in I_Delta =>
              (if Possible_I_Shapes (Cur_Piece.D) (YY, XX) then
                 Is_Empty (Cur_Board, Cur_Piece.Y + YY, Cur_Piece.X + XX))));

      for Y in I_Delta loop
         for X in I_Delta loop
            if Possible_I_Shapes (Cur_Piece.D) (Y, X) then
               Cur_Board (Cur_Piece.Y + Y) (Cur_Piece.X + X) := Cur_Piece.S;
            end if;
         end loop;
      end loop;

   when Three_Shape =>
      --  intermediate assertion needed for proof
      pragma Assert
        (for all YY in Three_Delta =>
           (for all XX in Three_Delta =>
              (if Possible_Three_Shapes (Cur_Piece.S, Cur_Piece.D) (YY, XX) then
                 Is_Empty (Cur_Board, Cur_Piece.Y + YY, Cur_Piece.X + XX))));

      for Y in Three_Delta loop
         for X in Three_Delta loop
            if Possible_Three_Shapes (Cur_Piece.S, Cur_Piece.D) (Y, X) then
               Cur_Board (Cur_Piece.Y + Y) (Cur_Piece.X + X) := Cur_Piece.S;
            end if;
         end loop;
      end loop;
end case;

With these intermediate assertions, GNATprove proves there is no possible array index out of bounds.

The situation is again different in Move, as there is no loop here. In fact, the decrements and increments in Move may indeed raise an exception at run time, if Move is called on a piece that is too close to the borders of the board. We need to prevent such errors by adding a precondition to Move that does not allow such inputs:

function Move_Is_Possible (P : Piece; A : Action) return Boolean is
   (case A is
      when Move_Left   => P.X - 1 in PX_Coord,
      when Move_Right  => P.X + 1 in PX_Coord,
      when Move_Down   => P.Y + 1 in PY_Coord,
      when Turn_Action => True);

function Move (P : Piece; A : Action) return Piece is
   (case A is
      when Move_Left   => P'Update (X => P.X - 1),
      when Move_Right  => P'Update (X => P.X + 1),
      when Move_Down   => P'Update (Y => P.Y + 1),
      when Turn_Action => P'Update (D => Turn_Direction (P.D, A)))
with
  Pre => Move_Is_Possible (P, A);

With this precondition, GNATprove proves there is no possible range check failure in Move, but it issues a message about a possible precondition failure when calling Move in Do_Action. We have effectively pushed the problem to Move's caller! We need to prevent calling Move in an invalid context by adding a suitable test:

procedure Do_Action (A : Action; Success : out Boolean) is
   Candidate : Piece;
begin
   if Move_Is_Possible (Cur_Piece, A) then
      Candidate := Move (Cur_Piece, A);

      if No_Overlap (Cur_Board, Candidate) then
         Cur_Piece := Candidate;
         Success := True;
         return;
      end if;
   end if;

   Success := False;
end Do_Action;

The program is now up to 181 sloc, a relatively modest increase. With this code modification, GNATprove proves in 28s that the integrity of the program is preserved: there are no possible run-time errors, and no precondition violations. See files tetris_integrity.ad? in the tetris_core.tgz archive attached. All timings on GNATprove are given with switches -j0 –prover=cvc4,altergo –timeout=20 on a 2.7 GHz Core i7 with 16 GB RAM.

Proving Functional Properties of Tetris Code

We would like to express and prove that the code of Tetris maintains the board in one of 4 valid states:

  • Piece_Falling: a piece is falling, in which case Cur_Piece is set to this piece
  • Piece_Blocked: the piece Cur_Piece is blocked by previous fallen pieces in the board Cur_Board
  • Board_Before_Clean: the piece has been included in the board, which may contain complete lines that need to be deleted
  • Board_After_Clean: complete lines have been deleted from the board

We define a type State that can have these 4 values, and a global variable Cur_State denoting the current state. We can now express the invariant that Tetris code should maintain:

function Valid_Configuration return Boolean is
   (case Cur_State is
      when Piece_Falling | Piece_Blocked => No_Overlap (Cur_Board, Cur_Piece),
      when Board_Before_Clean => True,
      when Board_After_Clean => No_Complete_Lines (Cur_Board))
with Ghost;

where No_Complete_Lines returns True if there are no complete lines in the board (that is, they have been removed and counted in the player's score):

function No_Complete_Lines (B : Board) return Boolean is
   (for all Y in Y_Coord => not Is_Complete_Line (B(Y)))
with Ghost;

Note the aspect Ghost on both functions, which indicates that these functions should only be called in assertions and contracts. Ghost code has been introduced for SPARK, but it can also be used independently of formal verification. The idea is that ghost code (it can be a variable, a type, a function, a procedure, etc.) should not influence the behavior of the program, and be used only to verify properties (either dynamically or statically), so the compiler can remove it when compiling without assertions. As you can expect, we also declared type State and variable Cur_State as Ghost:

type State is (Piece_Falling, Piece_Blocked, Board_Before_Clean, Board_After_Clean) with Ghost;

Cur_State : State with Ghost;

We add preconditions and postconditions to the API of Tetris, to express that they should maintain this invariant:

procedure Do_Action (A : Action; Success : out Boolean) with
  Pre  => Valid_Configuration,
  Post => Valid_Configuration;

procedure Include_Piece_In_Board with
  Global => (Input => Cur_Piece, In_Out => (Cur_State, Cur_Board)),
  Pre    => Cur_State = Piece_Blocked and then
            Valid_Configuration,
  Post   => Cur_State = Board_Before_Clean and then
            Valid_Configuration;
--  transition from state where a piece is falling to its integration in the
--  board when it cannot fall anymore.

procedure Delete_Complete_Lines with
  Global => (Proof_In => Cur_Piece, In_Out => (Cur_State, Cur_Board)),
  Pre    => Cur_State = Board_Before_Clean and then
            Valid_Configuration,
  Post   => Cur_State = Board_After_Clean and then
            Valid_Configuration;
--  remove all complete lines from the board

Note the presence of Valid_Configuration in precondition and in postcondition of every procedure. We also specify in which states the procedures should be called and should return. Finally, we add code that performs the state transition in the body of Include_Piece_In_Board:

Cur_State := Board_Before_Clean;

and Delete_Complete_Lines:

Cur_State := Board_After_Clean;

Although we're introducing here code to be able to express and prove the property of interest, it is identified as ghost code, so that the GNAT compiler can discard it during compilation when assertions are disabled.

GNATprove proves that Do_Action and Include_Piece_In_Board implement their contract, but it does not prove the postcondition of Delete_Complete_Lines. This is expected, as this subprogram contains two loops whose detailed behavior should be expressed in a loop invariant for GNATprove to complete the proof. The first loop in Delete_Complete_Lines deletes all complete lines, replacing them by empty lines. It also identifies the first such line from the bottom (that is, with the highest value in the Y axis) for the subsequent loop. Therefore, the loop invariant needs to express that none of the lines in the range already treated by the loop is complete:

for Del_Line in Y_Coord loop
   if Is_Complete_Line (Cur_Board (Del_Line)) then
      Cur_Board (Del_Line) := Empty_Line;
      Has_Complete_Lines := True;
      To_Line := Del_Line;
      pragma Assert (Cur_Board (Del_Line)(X_Coord'First) = Empty);
   end if;
   pragma Loop_Invariant
     (for all Y in Y_Coord'First .. Del_Line => not Is_Complete_Line (Cur_Board (Y)));
end loop;

The second loop in Delete_Complete_Lines shifts non-empty lines to the bottom of the board, starting from the deleted line identified in the previous loop (the line with the highest value in the Y axis). Therefore, the loop invariant needs to express that this loop maintains the property established in the first loop, that no line in the board is complete:

if Has_Complete_Lines then
   for From_Line in reverse Y_Coord'First .. To_Line - 1 loop
      pragma Loop_Invariant (No_Complete_Lines (Cur_Board));
      pragma Loop_Invariant (From_Line < To_Line);
      if not Is_Empty_Line (Cur_Board (From_Line)) then
         Cur_Board (To_Line) := Cur_Board (From_Line);
         Cur_Board (From_Line) := Empty_Line;
         To_Line := To_Line - 1;
         pragma Assert (Cur_Board (From_Line)(X_Coord'First) = Empty);
      end if;
   end loop;
end if;

Note that we added also two intermediate assertions to help the proof. GNATprove now proves the program completely (doing both flow analysis and proof with switch –mode=all) in 50s. See files tetris_functional.ad? in the tetris_core.tgz archive attached.

Obviously, proving that complete lines are deleted is only an example of what could be proved on this program. We could also prove that empty lines are all located at the top of the board, or that non-complete lines are preserved when deleting the complete ones. These and other properties of Tetris are left to you reader as an exercise! Now that the core game logic is developed, we can switch to running this game on the SAM4S Xplained Pro Evaluation Kit.

Developing a Board Support Package (BSP) for the Atmel SAM4S - Startup code

We already had a compiler targeting the ARM line of processors, including the variant present in the SAM4S - the Cortex-M4. But like many modern processors, the Atmel SAM4S is not simply a processor but a SOC (System On Chip). A SOC is a set of peripherals like memory controller, timers or serial controller around a core. Thus we needed to develop a Board Support Package to be able to run our Tetris program on the SAM4S.

In order to run a program, some of the devices must be initialized. The first device to initialize (and it should be initialized early) is the clock controller. Like all integrated circuits, the SAM4S needs a clock. To build cheap applications, the SAM4S provide an internal oscillator that deliver a 4MHz clock. But that clock is neither very precise nor stable, and has a low frequency. So we use an external 12MHz crystal on the main board and setup the internal clock multiplier (a PLL) to deliver a 120MHz clock, which provides much better performance than the default internal clock. This initialization is done very early before the initial setup of the Ada program (a.k.a. the program elaboration) so that the rest of the program is executed at full speed. Implementation-wise, that initialization sequence closely follows the sequence proposed by the Atmel manual, as this is very specific to the clock controller.

The second step is to adjust the number of wait states for accessing the flash memory. The flash memory is the non-volatile storage which contains the application. As accessing the flash memory is slower than the speed at which the processor runs, we need to add delays (also called wait states) for each access to the flash from the processor, and as we increase the speed of the processor we need to increase these delays. If we forget to insert these delays (this is simply a programmable parameter of the flash device), the processor will read incorrect values from the flash memory and crash very early. Fortunately the processor has a cache memory to reduce the number of flash memory accesses and therefore to limit the effect of the wait states.

The third step is to setup the watchdog peripheral. The purpose of a watchdog is to reinitialize the application in case of crashes. To signal that it is running, the application should periodically access the watchdog. Note that by proving that the core application in SPARK is free from run-time errors, we already reduce the number of potential crashes. So we chose here to disable the watchdog.

So what happens when the board is powered up?

The processor starts by executing the program contained in flash memory. The first instructions are assembly code that copy initialized variables (that could be later modified) from flash to RAM. This is done using the default setup and therefore at low speed (but the amount of bytes to copy is very low). Then it increases the number of wait states for the flash. At this time, the performance is very low, the lowest in this application. Then the clock device is programmed, which is a long sequence (the processor has to wait until the clock is stable). This process switches the frequency from 12MHz to 120MHz, thus increasing speed by a factor of 10! The application then disables the watchdog, and now that the processor is running at full speed, the program can start to execute.

Developing a BSP for the Atmel SAM4S - Simple Drivers

So we can execute a program on the board. This is a very good news, but we cannot observe any effect of this program: there is no communication yet with the outside world.

On a standard PC, a program interacts using the keyboard, the screen, the filesystem or the network. On that SAM4S Xplained Pro board, the program will get inputs via buttons, and it will produce visible effects by switching LEDs on and off and displaying patterns on the small OLED screen.

But there is no standard or predefined API for these devices. So we should write simple device drivers, starting with the simplest one: GPIO. GPIO is a common acronym used in the bare board world: General Purpose Input/Output. When configured as output, a GPIO pin can be commanded by a hardware register (in fact a variable mapped at a specific address) to emit current or not. On the board, four of them are interesting because they are connected to a LED. So if a 1 is written in a specific bit of the hardware register, the LED will be on and if a 0 is written it will be off. When configured as input, a GPIO register will reflect the value of a pin. Like for LEDs, four buttons are connected to a GPIO. So the application may read the status of a button from a register. A bit will be set to 1 when the button is pressed and set to 0 if not pressed.

The configuration of a GPIO device is highly device specific and we simply closely follow the vendor manual. First we need to power-up the device and enable its clock:

--  Enable clock for GPIO-A and GPIO-C

PMC.PMC_PCER0 := 2 ** PIOA_ID + 2 ** PIOC_ID;

Without that the device would be inactive. Technically, the LEDs and the buttons are connected to two different devices, GPIO-A and GPIO-C. Then we need to configure the port. Each bit in a GPIO word correspond to a GPIO pin. We need to configure pins for LEDs as output, and pins for buttons as input. We also need to enable that GPIO lines (there are more settings like pull-up or multi-driver, curious readers shall refer to the SAM4D data sheet):

--  Configure LEDs

PIOC.PER := Led_Pin_C + Led1_Pin_C + Led3_Pin_C
  + But2_Pin_C + But3_Pin_C;
PIOC.OER := Led_Pin_C + Led1_Pin_C + Led3_Pin_C;
PIOC.CODR := Led_Pin_C + Led1_Pin_C + Led3_Pin_C;
PIOC.MDDR := Led_Pin_C + Led1_Pin_C + Led3_Pin_C;
PIOC.PUER := But2_Pin_C + But3_Pin_C;

The device driver for a GPIO is often very simple. Here is a procedure that sets the state of LED 1:

procedure Set_Led1 (On : Boolean) is
begin
   if On then
      PIOC.CODR := Led1_Pin_C;
   else
      PIOC.SODR := Led1_Pin_C;
   end if;
end Set_Led1;

CODR means clear output data register; so any bit written as 1 will clear the corresponding bit in the output data register. Respectively, SODR means set output data register; so any bit written as 1 will set the corresponding bit in the output data register and bit written as 0 have no effect. This particular interface (one word to set and one to clear) is common and makes atomic bit manipulation easier.

The function to read the state of a button is also simple:

function Button3_Pressed return Boolean is
begin
   return (PIOC.PDSR and But3_Pin_C) = 0;
end Button3_Pressed;

We simply need to test a bit of the hardware register. Note that this is a low level driver. The code that calls this function must debounce the button. What does it mean? Real life is somewhat more complex than boolean logic. A button is a simple device with two pins. One is connected to power supply (here +3.0V) and the other is connected to the GPIO pin, but also to the ground via a resistor. When the button is not pressed, the voltage at the GPIO pin is 0V and the program will read 0 in the register. When the button is pressed, the voltage at the GPIO pin is +3.0V (well almost) and the program will read 1 in the register. But when the button is being pressed (this is not instantaneous), the voltage will increase irregularly from 0V to +3V, and due to capacity effects and non-uniformity of the button itself, the bit in the register will oscillate between 0 and 1 during a certain amount of time. If you don't take care, the application may consider a single push on the button as multiple pushes. Debouncing is avoiding that issue, and the simplest way to debounce is not reading the status again before a certain amount of time (like 20ms - but this depends on the button). We take care of that in the main loop of the Tetris game.

Developing a BSP for the Atmel SAM4S - OLED driver

Despite being very simple, the GPIO driver is a first good step. You can write simple applications to blink the LEDs or to switch the LEDs if a user presses a button. But LEDs and buttons are too simple for a Tetris game.

The SAM4S Xplained Pro has an OLED1 extension board. The OLED is a small screen and just behind it, there is a very small chip that controls the screen. This chip, the OLED controller, contains the bitmap that is displayed, refreshes periodically the screen, handles contrast, etc. These tasks are too complex to be performed by the CPU, hence the need for an OLED controller.

But this controller is somewhat complex. It communicates with the SAM4S using a dedicated interface, the SPI bus. SPI stands for Serial Peripheral Interface and as you can guess it uses a few lines (3 in fact) for a serial communication.

Let's have a closer look at the gory details. As you can suppose, we need to talk to the OLED controller whose reference is SSD1306. It accepts many commands, most of them to configure it (number of lines, number of columns, refresh rate) and some to change the bitmap (set the column or modify 8 pixels). All is done via the SPI so we need to first create an SPI driver for the SAM4S. Finally note that if commands can be sent to the OLED controller, that chip is not able to return any status.

The initialization of the SPI isn't particularly complex: we need to power on the device, and to configure the pins. In order to reduce the size of the chip and therefore its price, the SPI pins are multiplexed with GPIO pins. So we need to configure the pins so that they are used for SPI instead of GPIO. Here is an excerpt of the code for that (package Oled, procedure Oled_Configure):

--  Enable clock for SPI

PMC.PMC_PCER0 := PMC.PMC_PCER0 + 2 ** SPI_ID;

--  Configure SPI pins

PIOC.PER := Spi_DC_Pin_C + Spi_Reset_Pin_C;
PIOC.OER := Spi_DC_Pin_C + Spi_Reset_Pin_C;
PIOC.CODR := Spi_DC_Pin_C + Spi_Reset_Pin_C;
PIOC.MDDR := Spi_DC_Pin_C + Spi_Reset_Pin_C;
PIOC.PUER := Spi_DC_Pin_C + Spi_Reset_Pin_C;

Then there is a sequence to setup the SPI features: baud rate, number of bits per byte... We just have to read the Atmel documentation and set the right bits! (package Oled, procedure Spi_Init):

procedure Spi_Init is
   Baudrate : constant := 200; -- 120_000_000 / 5_000_000;
begin
   --  Reset SPI
   SPI.SPI_CR := SPI_CR.SWRST;

   --  Set mode register.
   --  Set master mode, disable mode fault, disable loopback, set chip
   --  select value, set fixed peripheral select, disable select decode.
   SPI.SPI_MR := (SPI.SPI_MR and not (SPI_MR.LLB or SPI_MR.PCS_Mask
                                        or SPI_MR.PS or SPI_MR.PCSDEC
                                        or SPI_MR.DLYBCS_Mask))
     or SPI_MR.MODFDIS or SPI_MR.MSTR or 0 * SPI_MR.DLYBCS;

   --  Set chip select register.
   SPI.SPI_CSR2 := 0 * SPI_CSR.DLYBCT or 0 * SPI_CSR.DLYBS
     or Baudrate * SPI_CSR.SCBR or (8 - 8) * SPI_CSR.BITS
     or SPI_CSR.CSAAT or 0 * SPI_CSR.CPOL or SPI_CSR.NCPHA;

   --  enable
   SPI.SPI_CR := SPI_CR.SPIEN;
end Spi_Init;

Now that the SPI interface is on, we need to configure the OLED controller. First action is to reset it so that it is in a known state. This is done via extra GPIO lines, and needs to follow a timing:

procedure Oled_Reset
is
   use Ada.Real_Time;
   Now : Time := Clock;
   Period_3us : constant Time_Span := Microseconds (3);
begin
   --  Lower reset
   PIOC.CODR := Spi_Reset_Pin_C;
   Now := Now + Period_3us;
   delay until Now;

   --  Raise reset
   PIOC.SODR := Spi_Reset_Pin_C;
   Now := Now + Period_3us;
   delay until Now;
end Oled_Reset;

Then we send the commands to configure the screen. That cannot be guessed and we'd better follow (again !) OLED controller manual:

procedure Ssd1306_Init is
begin
   --  1/32 duty
   Ssd1306_Cmd (SSD1306.CMD_SET_MULTIPLEX_RATIO);
   Ssd1306_Cmd (31);

   --  Set ram counter.
   Ssd1306_Cmd (SSD1306.CMD_SET_DISPLAY_OFFSET);
   Ssd1306_Cmd (0);

...

The OLED is now ready to use. The interface for it is rather low-level: we need to select the 'page' (which correspond to the line of the screen when oriented horizontally):

--  Set the current page (and clear column)
procedure Oled_Set_Page (Page : Unsigned_8);

And set pixels by groups of 8:

--  Draw 8 pixels at current position and increase position to the next
--  line.
procedure Oled_Draw (B : Unsigned_8);

Putting It All Together With Ravenscar

Most of the low-level work could be done (and was initially done) using the ZFP runtime of GNAT compiler. This runtime is very lightweight and doesn't offer tasking. But for bare boards, we have a better runtime called Ravenscar which provides the well-known Ravenscar subset of tasking features. We have already used it above for implementing delays, and we will also use it for the SPI driver. In the SPI driver, we need to send packet of bytes. We can send a new byte only when the previous one has been sent. With ZFP, we needed to poll until the byte was sent. But with Ravenscar, we can use interrupts and do something else (run another task) until the hardware triggers an interrupt to signal that the byte was sent. A protected type is declared to use interrupts:

protected Spi_Prot is
   pragma Interrupt_Priority (System.Interrupt_Priority'First);

   procedure Write_Byte (Cmd : Unsigned_8);

   procedure Interrupt;
   pragma Attach_Handler (Interrupt, Ada.Interrupts.Names.SPI_Interrupt);

   entry Wait_Tx;
private
   Tx_Done : Boolean := False;
end Spi_Prot;

The protected procedure 'Interrupt' is attached to the hardware interrupt via the pragma Attach_Handler, and the protected object is assigned to the highest priority via the pragma Interrupt_Priority (so that all interrupts are masked within the protected object to achieve exclusive access). The procedure Ssd1306_Write sends a command using the interrupt. It first programs the SPI device to send a byte and then waits for the interrupt:

procedure Ssd1306_Write (Cmd : Unsigned_8) is
begin
   Spi_Prot.Write_Byte (Cmd);

   Spi_Prot.Wait_Tx;
end Ssd1306_Write;

Writing a byte mostly consists of writing a hardware register (don't look at the PCS issue, that is an SPI low-level detail) and enabling SPI interrupts:

procedure Write_Byte (Cmd : Unsigned_8) is
begin
   --  Set PCS #2
   SPI.SPI_MR := (SPI.SPI_MR and not SPI_MR.PCS_Mask)
     or (SPI_MR.PCS_Mask and not ((2 ** 2) * SPI_MR.PCS));

   --  Write cmd
   SPI.SPI_TDR := Word (Cmd) * SPI_TDR.TD;

   --  Enable TXEMPTY interrupt.
   SPI.SPI_IER := SPI_SR.TXEMPTY;
end Write_Byte;

When the interrupt is triggered by the SPI device, the Interrupt procedure is called. It acknowledges the interrupt (by disabling it) and opens the entry (the Ssd1306_Write procedure is waiting on that entry):

procedure Interrupt is
begin
   if (SPI.SPI_SR and SPI_SR.TXEMPTY) /= 0 then
      --  Disable TXEMPTY interrupt
      SPI.SPI_IDR := SPI_SR.TXEMPTY;

      --  Set the barrier to True to open the entry
      Tx_Done := True;
   end if;
end Interrupt;

In the entry, we simply close the entry guard and clean the SPI state:

entry Wait_Tx when Tx_Done is
begin
   --  Set the barrier to False.
   Tx_Done := False;

   --  Deselect device
   SPI.SPI_MR := SPI.SPI_MR or SPI_MR.PCS_Mask;

   --  Last transfer
   SPI.SPI_CR := SPI_CR.LASTXFER;
end Wait_Tx;

Conclusion

You can see the results presented by our colleague Quentin at GNAT Pro Industrial User Day:

atmel board
board tetris
board tetris zoom
quentin board
quentin talk

Overall, it was one week effort to produce this demo of SPARK on ARM. The code is available in the tetris.tgz archive attached if you're interested in experimenting with formal verification using SPARK or development in Ada for ARM. The source code is bundled with the complete development and verification environment for Linux and Windows in the archive at http://adaco.re/85.

To know more about SPARK:

To know more about GNAT for ARM:

Some other people blogs show projects using GNAT on ARM.

Attachments

]]>
Welcome To AdaCore's Blog http://blog.adacore.com/welcome-to-adacores-blog Wed, 07 Jan 2015 07:00:00 +0000 Cyrille Comar http://blog.adacore.com/welcome-to-adacores-blog

Happy new Year everyone!

I'm proud, if not a bit nervous, to be the one firing the very first post on this brand new blog. Why are we starting a corporate blog at this time? There are many reasons for this. The main one is that there are many things happening at AdaCore and around the GNAT technology and we are seeking better ways to make them widely known while offering interested recipients the possibility to react and interact with us.

This blog is first a replacement for the GEMs program which has run for over 8 years and produced more than 150 entries sharing some of the more interesting and sometimes puzzling technical aspects of the varous versions of Ada and of the GNAT toolset. Contrary to the GEM program, we do not intend to limit this blog to purely technical posts. We also wish to share and discuss on less technical topics such as the advances of software static verification techniques, the business models associated with open source or the economical and human challenges in producing really safe and secure software at reasonnable cost.

Considering this last point, I have been impressed by the invasion of drone-related stories in the newspapers. There is hardly a week that passes without a mention of drones either in some kind of futuristic service announcements or potentially catastrophic usages. From bringing your bottle of milk to your doorstep to controlling the state of gaz pipelines in hostile environments, from unidentified flights over nuclear plants to abducted military drones. Whether we like it or not, drones are in the process of taking a significant place in our lifes and along with them, the software that control their flights and missions. A good question for us, who feel strongly about safety, is: if the drones have been a very nice vector for popularizing some of the aeronautics know-how, will we be able to popularize fast enough the "safe" programming know-how that has allowed aircrafts to claim a first place among safe transportation means?

Would answering "yes" to such a question only be yet-another-soon-to-be-forgotten new Year's resolution?

]]>