What features or architecture for a text answering system?

by ThePassenger   Last Updated August 23, 2018 14:19 PM

From this dataset with paragraphs, questions about these paragraphs and answers from these paragraphs, when there exists, I'm trying to predict the sentence where there is an answer. After unsupervised and supervised methods, which weren't really successful, I'm trying with neural networks which you can found in this GitHub repository. After splitting the sentences in a fixed size of ten I have created three features for each of the ten sentences : the cosine distance for each (question-sentence) algorithm, the euclidean one and root matching between the of the question and the roots of the sentences.

So I can have up to 30 inputs vectors in train_x, with the actual target sentence stored in train_y I tried to test two neural networks with 20 inputs: cosine-euclidean on the one hand and cosine-root on the other hand. With the following model:

# create model
model = Sequential()
model.add(Dense(12, input_dim=20, activation='relu'))
model.add(Dense(20, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
# Compile model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
# Fit the model
model.fit(train_x, train_y, epochs=150, batch_size=10)

But the accuracy rate is always quite slow and when I evaluate it with:

# evaluate the model
scores = model.evaluate(test_x, test_y)
print("\n%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))

The first one had an accuracy of 25,37% and the second one of acc: 26.33%

So I'm wondering who is the culprit: is the distance who even if it seems to be good at finding the answer when there is one in one sentence seems not to be a really good feature when there is none? Or is it the root matching feature or is it my neural network architecture ?

Related Questions

Correctness of a skewed cosine similarity graph

Updated April 11, 2018 12:19 PM

Cosine Similarity Intuition

Updated April 23, 2017 11:19 AM

Dispersion index and Cosine distance

Updated July 03, 2019 06:19 AM

How to combine Euclidean and Cosine distance?

Updated April 01, 2016 08:08 AM

Which similarity formula should I use?

Updated March 28, 2019 12:19 PM