[mary-dev] testing a NLP component
Marc Schroeder
marc.schroeder at dfki.de
Mon May 16 16:09:32 CEST 2011
Ah, now we can make progress. You are mixing Junit 3 and 4: your tests
are written according to Junit 4 (@Test ...), but your exception is
thrown by Junit 3 classes -- I think it is because you extend TestCase.
Try changing
public class preprocessorFRtest extends TestCase {
to
public class preprocessorFRtest {
And remove any import statements for junit.framework.... from the top of
your file.
Then run the junit runner again.
Best,
Marc
On 16.05.11 15:27, fxavier at ircam.fr wrote:
> So sorry, I found out that the the exception left pannel was hidden.
> Anyway, here's attached the screenshot.
>
>
>
>> weird. Can you send me a screenshot?
>> M.
>>
>> On 16.05.11 14:18, fxavier at ircam.fr wrote:
>>> Yes that's what I did, and I can't even get the red bar, all I have is
>>>
>>> Run 0/1 Errors:1
>>>
>>> Regards,
>>>
>>> Florent
>>>
>>>
>>>> Just a thought: you did change to the "junit" tab, did you? Expect to
>>>> see the exception on the left side of that tab.
>>>>
>>>> Best,
>>>> Marc
>>>>
>>>> On 05.05.11 18:34, fxavier at ircam.fr wrote:
>>>>> Yes I used eclipse, run as Junit test. Still nothing...
>>>>>
>>>>> Florent
>>>>>
>>>>>
>>>>>
>>>>>> Hi Florent,
>>>>>>
>>>>>> how did you run the test? Using "ant tests"? Then there should be an
>>>>>> xml
>>>>>> file containing the test results in folder log/test-reports/... and
>>>>>> normally, a server log file in log/junittest.mary.log...
>>>>>>
>>>>>> An alternative could be to run the test within Eclipse -- in
>>>>>> Eclipse's
>>>>>> package explorer, right click the test class, and select from the
>>>>>> context menu, "Run As"->"JUnit test". this should give you the
>>>>>> exception
>>>>>> immediately.
>>>>>>
>>>>>> Hope that helps. Keep asking, this is useful to sort out, and
>>>>>> document...
>>>>>>
>>>>>> Best,
>>>>>> Marc
>>>>>>
>>>>>> On 05.05.11 13:15, fxavier at ircam.fr wrote:
>>>>>>> Hi All,
>>>>>>>
>>>>>>> Thank you very much Marc, it helps a lot. I did everything you just
>>>>>>> mentionned.
>>>>>>>
>>>>>>> So if I understood well, adding those simple lines in the method
>>>>>>> shouldn't
>>>>>>> cause problems?
>>>>>>>
>>>>>>> //show output
>>>>>>> String output = new String(result.toString());
>>>>>>> System.out.println(output);
>>>>>>>
>>>>>>> Problem is, I don't receive anything in the console, and (with or
>>>>>>> without
>>>>>>> the added lines) I can't get the green tag proving the test has
>>>>>>> passed.
>>>>>>> I
>>>>>>> just have:
>>>>>>>
>>>>>>> Run 0/1 Errors:1
>>>>>>>
>>>>>>> No comment, nothing at all, and there are no errors in the code, so
>>>>>>> what
>>>>>>> is this error?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Florent
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> PS: so sorry I'm not able to find any solution on my own...
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Hi Florent,
>>>>>>>>
>>>>>>>> good start, but a few things need fixing.
>>>>>>>>
>>>>>>>> First, in order to test-run a MARY module, for most MARY modules
>>>>>>>> you
>>>>>>>> need to first start the entire MARY system. Also, for log4j to work
>>>>>>>> properly you need to configure it; so you need something like:
>>>>>>>>
>>>>>>>> @BeforeClass
>>>>>>>> public static void startMARY() throws Exception {
>>>>>>>> if (Mary.currentState() == Mary.STATE_OFF) {
>>>>>>>> Mary.startup();
>>>>>>>> }
>>>>>>>> if (!MaryUtils.isLog4jConfigured()) {
>>>>>>>> BasicConfigurator.configure();
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> This will avoid the warning message you got from log4j:
>>>>>>>>
>>>>>>>> > log4j:WARN No appenders could be found for logger
>>>>>>>> (marytts.IO).
>>>>>>>> > log4j:WARN Please initialize the log4j system properly.
>>>>>>>>
>>>>>>>>
>>>>>>>> Second, you are trying to set plain text for input format TOKENS.
>>>>>>>> But
>>>>>>>> TOKENS is a RAWMARYXML format, so when the XML parser tries to make
>>>>>>>> sense of your data, it fails:
>>>>>>>>
>>>>>>>> > [Fatal Error] :1:1: Content is not allowed in prolog.
>>>>>>>>
>>>>>>>> The way to avoid this is to enter proper XML data, either using
>>>>>>>> md.setData() or using md.readFrom(). I'll describe below how I
>>>>>>>> usually
>>>>>>>> do this.
>>>>>>>>
>>>>>>>>
>>>>>>>> Third, you are blinding yourself by discarding the helpful
>>>>>>>> exception.
>>>>>>>> NEVER EVER do something like this:
>>>>>>>>
>>>>>>>> > try
>>>>>>>> > {
>>>>>>>> > pr.process(md);
>>>>>>>> > }
>>>>>>>> > catch (Exception e)
>>>>>>>> > {
>>>>>>>> > return;
>>>>>>>> > }
>>>>>>>>
>>>>>>>> Take a look at some slides where I put down what I think are good
>>>>>>>> practices regarding exception handling:
>>>>>>>> http://mary.opendfki.de/repos/trunk/doc/ErrorHandling.pdf
>>>>>>>>
>>>>>>>>
>>>>>>>> Fourth, the test should verify that the processing result matches
>>>>>>>> expectations. A complex example like "M., 06.67.21.05.41, #, 423
>>>>>>>> Km,
>>>>>>>> 30
>>>>>>>> €. 20h14." will be difficult for this purpose; maybe here the
>>>>>>>> expected
>>>>>>>> state is not a specific outcome, but rather the fact that the
>>>>>>>> module
>>>>>>>> can
>>>>>>>> process this without crashing at all!
>>>>>>>>
>>>>>>>> One way how to write a test such that it is clean and readable is
>>>>>>>> to
>>>>>>>> write it backwards, starting with the verification... for example:
>>>>>>>>
>>>>>>>> @Test
>>>>>>>> public void canProcessWildStuff() throws Exception {
>>>>>>>>
>>>>>>>> ...
>>>>>>>> // verify expected result:
>>>>>>>> assertNotNull(result);
>>>>>>>> }
>>>>>>>>
>>>>>>>> Then, how did we get there:
>>>>>>>>
>>>>>>>>
>>>>>>>> @Test
>>>>>>>> public void canProcessWildStuff() throws Exception {
>>>>>>>>
>>>>>>>> ...
>>>>>>>> // exercise system under test:
>>>>>>>> MaryData result = preprocessor.process(input);
>>>>>>>> // verify expected result:
>>>>>>>> assertNotNull(result);
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>> and finally, set up the system under test:
>>>>>>>>
>>>>>>>> @Test
>>>>>>>> public void canProcessWildStuff() throws Exception {
>>>>>>>> // Set up system under test:
>>>>>>>> MaryModule preprocessor =
>>>>>>>> ModuleRegistry.getModule(marytts.language.de.Preprocess.class);
>>>>>>>> MaryData input = new MaryData(MaryDataType.TOKENS,
>>>>>>>> Locale.GERMAN);
>>>>>>>> input.readFrom(this.getClass().getResourceAsStream("wildStuff.tokens"));
>>>>>>>> // exercise system under test:
>>>>>>>> MaryData result = preprocessor.process(input);
>>>>>>>> // verify expected result:
>>>>>>>> assertNotNull(result);
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>> In this example, you see that I am reading the XML document to test
>>>>>>>> from
>>>>>>>> a classpath resource, in the same package as the test class, called
>>>>>>>> "wildStuff.tokens". How to create that document? Well, assume we
>>>>>>>> have
>>>>>>>> components that can create this input format for the target
>>>>>>>> language.
>>>>>>>> For German, this is the case for the public demo, so we can go to
>>>>>>>> http://mary.dfki.de:59125/documentation.html#synthesis and enter
>>>>>>>> into
>>>>>>>> the GET example:
>>>>>>>> INPUT_TEXT: M., 06.67.21.05.41, #, 423 Km, 30 €. 20h14.
>>>>>>>> INPUT_TYPE: TEXT
>>>>>>>> OUTPUT_TYPE: TOKENS
>>>>>>>> LOCALE: de
>>>>>>>> ... and click "Submit Query". If that works, you should get a
>>>>>>>> response
>>>>>>>> containing your test document (see below); copy that into file
>>>>>>>> "wildStuff.tokens" in the test package.
>>>>>>>>
>>>>>>>> Hope this helps. I know it's complex, but I think it's worth doing
>>>>>>>> these
>>>>>>>> things properly. It will pay off after the initial investment, I
>>>>>>>> think.
>>>>>>>>
>>>>>>>>
>>>>>>>> Good luck!
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Marc
>>>>>>>>
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Mary-dev mailing list
>>>>>>> Mary-dev at dfki.de
>>>>>>> http://www.dfki.de/mailman/cgi-bin/listinfo/mary-dev
>>>>>>
>>>>>> --
>>>>>> Dr. Marc Schröder, Senior Researcher at DFKI GmbH
>>>>>> Project leader for DFKI in SSPNet http://sspnet.eu
>>>>>> Team Leader DFKI TTS Group http://mary.dfki.de
>>>>>> Editor W3C EmotionML Working Draft http://www.w3.org/TR/emotionml/
>>>>>> Portal Editor http://emotion-research.net
>>>>>>
>>>>>> Homepage: http://www.dfki.de/~schroed
>>>>>> Email: marc.schroeder at dfki.de
>>>>>> Phone: +49-681-85775-5303
>>>>>> Postal address: DFKI GmbH, Campus D3_2, Stuhlsatzenhausweg 3, D-66123
>>>>>> Saarbrücken, Germany
>>>>>> --
>>>>>> Official DFKI coordinates:
>>>>>> Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH
>>>>>> Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany
>>>>>> Geschaeftsfuehrung:
>>>>>> Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender)
>>>>>> Dr. Walter Olthoff
>>>>>> Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes
>>>>>> Amtsgericht Kaiserslautern, HRB 2313
>>>>>> _______________________________________________
>>>>>> Mary-dev mailing list
>>>>>> Mary-dev at dfki.de
>>>>>> http://www.dfki.de/mailman/cgi-bin/listinfo/mary-dev
>>>>>>
>>>>>
>>>>
>>>> --
>>>> Dr. Marc Schröder, Senior Researcher at DFKI GmbH
>>>> Project leader for DFKI in SSPNet http://sspnet.eu
>>>> Team Leader DFKI TTS Group http://mary.dfki.de
>>>> Editor W3C EmotionML Working Draft http://www.w3.org/TR/emotionml/
>>>> Portal Editor http://emotion-research.net
>>>>
>>>> Homepage: http://www.dfki.de/~schroed
>>>> Email: marc.schroeder at dfki.de
>>>> Phone: +49-681-85775-5303
>>>> Postal address: DFKI GmbH, Campus D3_2, Stuhlsatzenhausweg 3, D-66123
>>>> Saarbrücken, Germany
>>>> --
>>>> Official DFKI coordinates:
>>>> Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH
>>>> Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany
>>>> Geschaeftsfuehrung:
>>>> Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender)
>>>> Dr. Walter Olthoff
>>>> Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes
>>>> Amtsgericht Kaiserslautern, HRB 2313
>>>>
>>>>
>>>
>>
>> --
>> Dr. Marc Schröder, Senior Researcher at DFKI GmbH
>> Project leader for DFKI in SSPNet http://sspnet.eu
>> Team Leader DFKI TTS Group http://mary.dfki.de
>> Editor W3C EmotionML Working Draft http://www.w3.org/TR/emotionml/
>> Portal Editor http://emotion-research.net
>>
>> Homepage: http://www.dfki.de/~schroed
>> Email: marc.schroeder at dfki.de
>> Phone: +49-681-85775-5303
>> Postal address: DFKI GmbH, Campus D3_2, Stuhlsatzenhausweg 3, D-66123
>> Saarbrücken, Germany
>> --
>> Official DFKI coordinates:
>> Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH
>> Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany
>> Geschaeftsfuehrung:
>> Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender)
>> Dr. Walter Olthoff
>> Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes
>> Amtsgericht Kaiserslautern, HRB 2313
>>
>>
--
Dr. Marc Schröder, Senior Researcher at DFKI GmbH
Project leader for DFKI in SSPNet http://sspnet.eu
Team Leader DFKI TTS Group http://mary.dfki.de
Editor W3C EmotionML Working Draft http://www.w3.org/TR/emotionml/
Portal Editor http://emotion-research.net
Homepage: http://www.dfki.de/~schroed
Email: marc.schroeder at dfki.de
Phone: +49-681-85775-5303
Postal address: DFKI GmbH, Campus D3_2, Stuhlsatzenhausweg 3, D-66123
Saarbrücken, Germany
--
Official DFKI coordinates:
Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH
Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany
Geschaeftsfuehrung:
Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender)
Dr. Walter Olthoff
Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes
Amtsgericht Kaiserslautern, HRB 2313
More information about the Mary-dev
mailing list