[Rock-dev] Questions Regarding Typelib

[Sylvain Joyeux]

Oh ... I would also need the size of each stream's sample (including the
size of vectors if there are any) ...

About index loading: the way the index was marshalled needed to be changed
(but was not) after the change you made to indexing (i.e. making indexes
dense). A 3-line patch improves performance quite a lot already. Alignment
is already pretty good on my test file (~4s).

I've generated a 6G log file with 3 streams, ~650k samples in 1 minute of
logical time, reading the streams using the Ruby stream aligner yields
~115x if I don't read the data sample, ~8x if I do. To me, this definitely
means that the performance problem is not pocolog's low-level handling but
both I/O (I have "images" of 1M, so I/O is actually a big performance
problem), typelib demarshalling (there is one thing in this code path that
sucks big time currently in the typelib-ruby binding related to
demarshalling, did not measure the actual impact of it, though) and the
fact that I hit the ruby GC pretty hard by creating 650k samples and 650k
intermediate buffers.

The point is: as soon as you'll get ruby bindings, you'll have the same
issues (GC, typelib demarshalling and I/O will not go away). The only thing
that goes away is the creation of 650k intermediate buffers, but that can
be removed with very little work already.

Now, I do get 8x replay speed when demarshalling the data. That's very far
from 0.6. When you talk about bad performance, were you using the replay UI
?

For reference:

pocolog test.0.log < this is a 5.9G file

Stream images [/images] < these are 1M samples
  5994 samples from Sun 08/06/2014 22:36:42 to Sun 08/06/2014 22:46:41
[0:09:59.299]
Stream motors [/motors] < these are 100 bytes samples
  599708 samples from Sun 08/06/2014 22:36:42 to Sun 08/06/2014 22:46:41
[0:09:59.706]
Stream pose_samples [/pose_samples] < these are 512 bytes samples
  59970 samples from Sun 08/06/2014 22:36:42 to Sun 08/06/2014 22:46:41
[0:09:59.690]

Sylvain
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.dfki.de/pipermail/rock-dev/attachments/20140608/c3404cdd/attachment.htm