Performance metrics to measure the validity of predicted text?

by Emna Jaoua   Last Updated June 21, 2018 15:19 PM

I have an RNN word based model. The goal of this model is to predict a certain number of words (5000 in this case) given a seed input (15 words). In my case, the words represent xml markups. For example the vocabulary can have [ '<Virtual>', '<Width>', '<XRefMasterInterface>', '<XRefRegMemSet>', '<XRefSignal>', '<XRefTargetID>', '<component>', '<html>', 'AC Detection enable', 'AC detection unit control', 'AC period over value'] ...

I managed to run successfully the model but I want to measure the performance of my model as well as the validity of the generated text. This is an exmaple of the generated text:

</Name><ShortDescription> llc_trig1</ShortDescription><ConstDefBlock>2720</AddressUnit><DataUnit></ResetMask><Port><Custom></Custom><Hidden> false</Hidden><ID>581</ID><LongDescription><html></html></LongDescription><Name> col0_8</Name><ShortDescription> disable</ShortDescription><Value>0</Value></EnumerationElement></BitFieldElement><BitFieldElement><Custom></Custom><Hidden> false</Hidden><ID>60</ID><LongDescription><html></html></LongDescription><Name> res0</Name><ShortDescription></ShortDescription><Offset>16</Offset><Width>1</Width><AccessExternal><AccessType> read</AccessType><StrobeType>NoStrobe</StrobeType></AccessExternal><AccessInternal><AccessType> write</AccessType><StrobeType>NoStrobe</StrobeType></AccessInternal><MaxVal>0</MaxVal><MinVal>0</MinVal><Reserved> unreserved</Reserved><SignInterpretation> unsigned</SignInterpretation><Virtual> false</Virtual><DefaultValue>0</DefaultValue><DefaultMask>0</DefaultMask><EnumerationElement><Custom></Custom>

While searching through the internet, I found the RMSE measure for regression tasks. Can it be used in my case ?



Related Questions




When to use different kinds of performance metrics

Updated December 10, 2017 04:19 AM

Which is the correct F-beta-score formula?

Updated March 12, 2018 19:19 PM