[Rock-dev] Problems with C++ constructors and RTEMS

[Joel Sherrill]

On Fri, Sep 16, 2011 at 4:45 AM, Anita <ana.vazquez.alonso at gmail.com> wrote:
> Hi,
>
> I managed to compile and link all the dependencies libraries together
> with RTEMS 4.10.1 and Newlib, producing a final binary of about 32Mb.

That's rather large.  Is that the output of "size" or the file size (e.g. ls -l)

> I run it with QEMU but a problem in  runtime arose:
>
> The problem happens inside a c++ constructor, called from
> __do_global_ctors_aux (which if I understand it correctly, basically
> call all the constructors from the table of pointers to constructor
> defined in the .ctors section (i386-rtems-objdump -s -j .ctors
> message_producer_test | vim -)).
>
> __do_global_ctors_aux is called from RTEMS (_Thread_Handler
> ->INIT_NAME -> _init).
>
> The constructor is part of omniORB, omni_mutex::omni_mutex
> (http://pastebin.com/Pcd50Bni). Inside this constructor, a call to
> pthread_mutex_init is produced.
>
> From the gdb log (http://pastebin.com/QrRxWxse) I fear that the
> problem is allocating resources inside pthread_mutex_init, actually in
> objectallocate.c -> chainget.c -> chain.inl.
>
> At first, I though the problem was related to RTEMS running out of
> memory (and even tried to increase QEMU memory), but after taking a
> look to chain.inl, I think that the problem may be the omni_mutex
> constructor calling pthread_mutex_init and therefore allocating
> resources before RTEMS is completly ready. Although, RTEMS is the one
> calling __do_global_ctors_aux.
>
> I hope any of you can spread some light. In the meantime, what I have
> in mind is modify omniORB and delay the pthread_mutex_init call in
> omni_mutex.

RTEMS != Linux. :-D

I am sure that didn't help at all but it is a lead in to try to explain
the philosophical difference between RTEMS and Linux that
lead you to get this error.   The error is in fact an out of resources
error.  RTEMS was designed for to target systems with hard
resource requirements.  In this design view, it is better to preallocate
as much as possible so you don't have to deal with running out
of resources at run-time.  This makes the resulting system
safer and less likely to have a weird failure mode in this situation.

In RTEMS you configure the maximum number of each type of
object you want.  The defaults tend to be 0.  Memory is reserved
for RTEMS separate from the C Program Heap based upon your
cofniguration requests.  The sample in testsuites/samples/ticker
has the following configuration in the file system.h

============================================
#include <bsp.h> /* for device driver prototypes */

#define CONFIGURE_APPLICATION_NEEDS_CLOCK_DRIVER
#define CONFIGURE_APPLICATION_NEEDS_CONSOLE_DRIVER
#define CONFIGURE_MAXIMUM_TASKS             4
#define CONFIGURE_RTEMS_INIT_TASKS_TABLE
#define CONFIGURE_EXTRA_TASK_STACKS  (3 * RTEMS_MINIMUM_STACK_SIZE)

#include <rtems/confdefs.h>
============================================

The application says is needs a console (stdio) and clock tick (time
passage) device drivers.  It has 4 Classic API tasks maximum.
It is using a Classic API style initialization task -- the alternative
is a POSIX Threads initialization thread.  And each of the tasks
it is creating has a stack larger than the minimum required.  So
we reserve some extra memory for those.  See the rtems_task_create()
calls in init.c for the stack sizing.

If you looked at the hello world sample, it wouldn't have the Clock
driver and would only have one task.

In your case, you need to define CONFIGURE_MAXIMUM_POSIX_MUTEXES
to however many is required.

With all that background on the hard limit focus on RTEMS configuration,
RTEMS also has an "unlimited object mode" and "unified workspace" option.
This is probably more useful for you at this stage.  This lets you configure
that you want a potentially unlimited number of a class of objects and that
you want RTEMS Workspace and the C Program Heap to be the same
pool of memory.

#define CONFIGURE_MAXIMUM_POSIX_MUTEXES \
   rtems_resource_unlimited( 5 )
#defined CONFIGURE_MAXIMUM_POSIX_CONDITION_VARIABLES \
   rtems_resource_unlimited( 5 )
#define CONFIGURE_UNIFIED_WORK_AREAS

That says that POSIX mutexes and condition variables are "unlimited"
but the set is extended 5 instances at a time.  Create the sixth and
instances 6-10 will be added to the inactive pool for that object class.

The full set configuration macros are (hopefully) well documented here:

http://rtems.org/onlinedocs/doc-current/share/rtems/html/c_user/c_user00416.html

Also .. any time you see a call from _Objects_Allocate() fail, it is either
a maximum objects issue or not accounting for variable memory like
stack space or message buffers which must be allocated when the
object instance is created.

Since you are porting a software package, let me throw out another
thought.  There is likely a fixed base set of objects the package creates
such as for global mutexes or message queues.  A user of package X
will create instances of objects that it defines.   So if the package X has
a macaroni object that requires a mutex, condition variable, and
a message queue, then you can let the end user of package X know that
for each macaroni instance, they need to reserve A, B, and C.  For certain
cases, like the tasking in Ada and Go, we provide higher level configuration
helpers like CONFIGURE_MAXIMUM_ADA_TASKS to encapsulate this
configuration information.

I know my answer is a bit over the top but I want you to really understand this
part of RTEMS.  Figuring out how many of each kind of object when porting a
software package is always a challenge.  Embedded developers focused on
safe, reliable systems don't like surprises and using various techniques to
avoid running out of resources at run-time is a big part of it.  It is
not uncommon
for malloc() to be forbidden after system initialization.

FWIW I would like to clean this up and make it a blog entry for RTEMS.
So if anything is unclear, let's make it clear to you. :)

--joel sherrill
RTEMS


> Kind regards,
> Ana
>
> P.S. In order to simplify the understanding of the problem, I attach
> the code from mutexinit.c (http://pastebin.com/cC6G2UjF), and since
> _Thread_Disable_dispatch(), _POSIX_Mutex_Allocate() and
> _Thread_Enable_dispatch() are inline functions, I also attached the
> unrolled version (http://pastebin.com/MxwXCCj5).
>