[Rock-dev] Typekit type specialization

Steffen Planthaber steffen.planthaber at dfki.de
Tue Mar 9 08:54:27 CET 2021 

Hi,

Typelib cannot serialize private data or special data types etc, only 
POD types.

The standard way to solve this would be using opaque types, where you 
specify another (POD) type where the data is copied to and which is then 
send via port.

This involves a copy (no memory allocation) from the internal type to 
the type actually send and vice versa. (see 
https://github.com/rock-core/base-orogen-types/blob/master/base.orogen 
as a reference), but this is quite similar to a custom serialization in 
the end, but then the newly filled struct is serialized again by typelib.

When you don't need introspection on port data level (aka. look at 
values using rock-display) you also have the option to send custom 
serialized data using /std/vector<char> as type for the port and 
serialize manually. But doing this you loose type savety on port 
connection level.

I'd go for using opaques with structs that contain a 
std::vector<uint8_t> as data buffer for each (self-)serialized type and 
perhaps some meta-information for rock-display and then use custom 
serialization to fill the buffer. By using one struct per type you can 
keep type savety for connections and also minimize copying data as each 
value in the struct is only serialized once some by typekit and some by 
your serialization (Execpt perhaps additional metadata for rock-display).

If you have a lot of types affected, have a look on Saschas approach on 
using a opaque generating orogen plugin. The plugin automatically 
generates opaques using boost serialization, the transport type is type 
serialized by typekit, but has a large binary buffer, so the 
serialization for typekit is minimal and the boost serialization data 
had not to be serialized again.

https://github.com/envire/envire-orogen-envire_orogen

used here:

https://github.com/envire/envire-orogen-envire_maps/blob/master/envire_maps.orogen

When using custom serialization and you need introspection you can still 
write vizkit plugins for the port type and do a de-serialization in the 
visualizer, only the "off the shelf vizualization" with rock-dispaly 
doesn't work.

Hope you are well,

Steffen


Am 08.03.21 um 18:59 schrieb Goldhoorn Matthias (XC-AD/EFB1-NA):
> Hello Rock-Devs,
>
> long time not seen :).
> I currently play around with typekit and orogen in one of our projects.
> I did the basic build system integration and if we continue using it, 
> i might need to extend orogen (castxml handling) with cmake support 
> soon, but this is a differnt story.
>
> So, I currently struggle to create bindings/serialization helper for 
> some special types we use:
>
> 1) Typelib[WARN]: 
> /home/a/workspace/athena/build/vfc/include/vfc/memory/vfc_fixedblock_allocator.hpp:198: 
> cannot create the /vfc/TFixedBlockAllocator<SOMETHING>/storage_type 
> typedef, as it points to  which is ignored
> 2) Typelib[WARN]: 
> /home/a/workspace/athena/build/vfc/include/vfc/memory/vfc_fixedblock_allocator.hpp:89: 
> ignoring /vfc/TFixedBlockAllocator<SOMETHING,200> since its field 
> m_memblock is of the ignored type 
> /vfc/TFixedBlockAllocator<SOMETHING,200>/storage_type
> 3) Typelib[WARN]: 
> /home/a/workspace/athena/build/vfc/include/vfc/container/vfc_fixedvector.hpp:453: 
> ignoring /vfc/TFixedVector<SOMETHING,200,SOMETHING,200>>/storage_type 
> as it has a non-public access specifier: private
> 4) Typelib[WARN]: 
> /home/a/workspace/athena/build/vfc/include/vfc/container/vfc_fixedvector.hpp:93: 
> ignoring 
> /vfc/TFixedVector<SOMETHING,200,/vfc/TFixedBlockAllocator<SOMETHING,200>> 
> since its field m_memory is of the ignored type 
> /vfc/TFixedVector<SOMETHING,200,/vfc/TFixedBlockAllocator<SOMETHING,200>>/storage_type
>
> And i see several issues in:
> 1) there is a "points to <MISSING?> which is ignored" so obvisously 
> here is something wrong in the error message (and i havwe to trace 
> down the /storage_type which i have no clue where its comming from
> 2,3) private fields, normal error, see below....
> 4) simliar to one
>
> To solve the issues I thought provide a own 
> serialization/deserialization method to prevent the usage inside RTT.
>
> I wanted to go for a "specialize" of the types. However adding the the 
> following specialize to my .orogen project:
>
> ------------------------------------------------------------------------
> Typelib.specialize '/vfc/TFixedBlockAllocator<>' do
>      include Typelib::ContainerType::StdVector #something should hopefully not YET madder, to be written later
>    end
>    Typelib.specialize '/vfc/TFixedVector<>' do
>      include Typelib::ContainerType::StdVector #something should hopefully not YET madder, to be written later
>    end
>
> ------------------------------------------------------------------------
> does not have any effect. Same as if i would add it to typelib directly:
>
>
> ------------------------------------------------------------------------
> diff --git a/bindings/ruby/lib/typelib/standard_convertions.rb b/bindings/ruby/lib/typelib/standard_convertions.rb
> index fb3e3fb..f917070 100644
> --- a/bindings/ruby/lib/typelib/standard_convertions.rb
> +++ b/bindings/ruby/lib/typelib/standard_convertions.rb
> @@ -59,6 +59,10 @@ def concat(other_string)
>           include Typelib::ContainerType::StdVector
>       end
>   
> +    specialize '/vfc/TFixedVector<>' do
> +        include Typelib::ContainerType::FixedVector #something should hopefully not YET madder, to be written later
> +    end
> +
>       ####
>       # C string handling
>       if String.instance_methods.include? :ord
>
> ------------------------------------------------------------------------
>
> So my questions are:
> * Since i am out of rock, am i going the right way, why the specialize 
> is not taken into account?
> ** Cant it bypass the private fields, do i have to change my 
> underlaying core datatypes (if so are there other ways which are copy 
> free?, I would like to prevent going down this path)
> * Can I prevent a copy this way, or will the data strucutres be copied 
> if i go for a "specialize"?I
> Thanks for your help
>
> Best and stay safe,
> Matthias
>
> _______________________________________________
> Rock-dev mailing list
> Rock-dev at dfki.de
> https://www.dfki.de/mailman/cgi-bin/listinfo/rock-dev

-- 
  Steffen Planthaber

  Besuchsadresse der Nebengeschäftstelle:
  DFKI GmbH
  Robotics Innovation Center
  Robert-Hooke-Straße 5
  28359 Bremen, Germany

  Postadresse der Hauptgeschäftsstelle Standort Bremen:
  DFKI GmbH
  Robotics Innovation Center
  Robert-Hooke-Straße 1
  28359 Bremen, Germany

  Tel.:     +49 421 178 45-4125
  Zentrale: +49 421 178 45-0
  Fax:      +49 421 178 45-4150 (Faxe bitte namentlich kennzeichnen)
  E-Mail:   Steffen.Planthaber at dfki.de

  Weitere Informationen: http://www.dfki.de/robotik

   -------------------------------------------------------------
   Deutsches Forschungszentrum für Künstliche Intelligenz GmbH
   Trippstadter Strasse 122, 67663 Kaiserslautern, Germany

   Geschäftsführung:
   Prof. Dr. Antonio Krüger

   Vorsitzender des Aufsichtsrats:
   Dr. Gabriël Clemens
   Amtsgericht Kaiserslautern, HRB 2313
   -------------------------------------------------------------

-------------- nächster Teil --------------
Ein Dateianhang mit HTML-Daten wurde abgetrennt...
URL: https://www.dfki.de/pipermail/rock-dev/attachments/20210309/6a89ca86/attachment-0001.htm

More information about the Rock-dev mailing list