[mary-users] RES: MaryTTS Viseme data

Mon Apr 17 14:41:42 CEST 2017

Dear Joan Pere Sanchez,

Let explain what we have and where we´d like to go.

1)      WHAT WE HAVE

a.      Web Software solution with Voice Sintetizer and an Avatar with have specifics motions known by person with have Hearing Problems and understands (LIBRAS) (Brazilian Sign Language). Our Voice Sintentizer is used for who have visual system problems and other.

2)      WHERE WE´D LIKE TO GO.

a.      Motion detection Systems:-

                                                    i.     It will allow person with have Hearing Problems communicate with normal peoples. It system will detect LIBRAS symbols and will convert them to words that will played by TTS.

b.      Natural Voice Detection System:

                                                    i.     It will allow person with have Hearing Problems looks to an Avatar with will reply in LIBRAS symbols what persons are talking.

1.      Normal Class Rooms.

2.      Watch Movies.

3.      Detailed voice to text conversion for business.

4.      And much more.

c.      Local Voice Translation.

                                                    i.     Brazil is a white country with different accents. In some cases, is important talk with correct local accent. It allow easeful understanding what is talking.

d.      TTS auto detecting and talking correct Pontuation, exclamations....

                                                    i.     The TTS solutions I know, just talk as reading an text. By the way, in some cases it need to compose phrasys with “?,!...” that need some special manner to reproduce it sounds.  This would allow the listener to fell that he was talking to a real person.

Important:  Almost all solutions I talked above exists, but, still very expansive to our market.

Regards,

Oliveira/Jose Carlos de

Pessoal

Brasilia - DF - Lago Norte

SHIN CA05 CJM02 AP309

   cel:+5561-99311.9226

De: Joan Pere Sanchez [mailto:kaiserjp at gmail.com] 
Enviada em: segunda-feira, 17 de abril de 2017 07:28
Para: Jose Carlos de Oliveira
Cc: idoor; mary-users at dfki.de
Assunto: Re: [mary-users] MaryTTS Viseme data

Dear Jose Carlos,

I would like to know a bite more your work, if it's possible. I ask for because I'm planning to start try to generate 'personalized' voices for impaired people. So they can have some reinforcement having their own voice as a identity element. Also with my lip-sync method, I think it is possible to help blind people or visual impaired to 'read'  the words better avoiding the McGurk effect while they are using interfaces.
Well, you know, scientific curiosity only.

Thanks!!

2017-04-16 15:58 GMT+02:00 Jose Carlos de Oliveira <oliveirakol at gmail.com>:

Sorry!

I desired to redirect your message to my development group and made it wrong.

By the way, your PhD thesis is very good.

Here in Brazil we are working with MaryTTS since 2016 because and other TTS´s for few years.

The reason is that we developed a software called RYBENA to help Brazilian with have visual or hearth problems. You can see it in (Brazilian) site www.rybena.com.br 

For years we are using a very expansive TTS software for (Brazilian Portuguese) Developed by CPqD. 

So we are developing our own TTS using MaryTTS, because we found other with didn’t gave good voice sintetisys.

Regards, 

Oliveira/Jose Carlos de

Pessoal

Brasilia - DF - Lago Norte

SHIN CA05 CJM02 AP309

   cel:+5561-99311.9226

De: idoor [mailto:idoorlab88 at gmail.com] 
Enviada em: domingo, 16 de abril de 2017 10:28
Para: Jose Carlos de Oliveira
Cc: Joan Pere Sanchez; mary-users at dfki.de

Assunto: Re: [mary-users] MaryTTS Viseme data

Sorry, I do not know what is "Pode ajudar", is it Spanish?

Best regards

On Sun, Apr 16, 2017 at 9:21 AM, Jose Carlos de Oliveira <oliveirakol at gmail.com> wrote:

Pode ajudar

Oliveira/Jose Carlos de

Pessoal

Brasilia - DF - Lago Norte

SHIN CA05 CJM02 AP309

   cel:+5561-99311.9226 <tel:+55%2061%2099311-9226> 

De: mary-users-bounces at dfki.de [mailto:mary-users-bounces at dfki.de] Em nome de Joan Pere Sanchez
Enviada em: sábado, 15 de abril de 2017 15:05
Para: idoor Du
Cc: mary-users at dfki.de
Assunto: Re: [mary-users] MaryTTS Viseme data

Hi Dave,

This task is the main goal of my PhD thesis. I'm doing lip-sync from the input text over the time duration estimation done while the speech is generated. You can develop your own strategy for lip/mouth synchronization, but often this is an avatar (or interface -I'm using a talking head too-) dependent task. So, if you are using an avatar, it depends if you can use blend shapes to mix by interpolation from the initial pose to the next one. Most of MPEG-4 systems are able to do that automatically.

On one hand, you have each phoneme and their start and finish time. On the other hand, you can adjust a set of visemes for each basic expression (no more than 15 are needed) and then choose the sequence corresponding to each word you are generating. It's the more efficient and simple way to have an effective lip synchronization.

Don't hesitate to contact me if you want more info or refs about.

Bes regards,

2017-04-15 18:27 GMT+02:00 idoor Du <idoorlab88 at gmail.com>:

Hi all,

I am new to MaryTTS, tried to call its API via:

AudioInputStream audio = mary.generateAudio("testing");

Now I want to animate mouth/lip shapes at runtime based on the audio sound, how to achieve that? are there any viseme data associated with the audio?

Thanks in advance.

Dave

_______________________________________________
Mary-users mailing list
Mary-users at dfki.de
http://www.dfki.de/mailman/cgi-bin/listinfo/mary-users

-- 

Joan Pere Sànchez Pellicer

kaiserjp at gmail.com

www.chamaleon.net
+34 625 012 741 <tel:+34%20625%2001%2027%2041> 

-- 

Joan Pere Sànchez Pellicer

kaiserjp at gmail.com

www.chamaleon.net
+34 625 012 741

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.dfki.de/pipermail/mary-users/attachments/20170417/ab1bd60d/attachment-0001.htm 