Poor results (wrong dataset?)

[bazuker]

Hi,
I followed the guide in the video & jupyter notebook and trained my model based on the dataset from kaggle that were mention in the video. I tried all of them and spent several hours training the model but results are pretty bad no matter what dataset I used.
What dataset did you actually use to achieve accuracy shown in the screenshot from the jupyter notebook?

My typical results:

[Huub79]

I am having the same problem...
Any suggestions?

[akupf]

I´ve got the same result... any ideas?

[Huub79]

I added the following lines, before building the model:

from keras import backend as K
K.clear_session()

That helped

[bazuker]

@Huub79 did it actually improve the end results?

[Huub79]

No... Now I remember... For some reason you need to reload the weights after training....??

I added the load_network line to cell 7:

data_gen_test = dl.generate_clean_data(
    configs['data']['filename_clean'],
    batch_size=configs['data']['batch_size'],
    start_index=ntrain
)

ntest = nrows - ntrain
steps_test = int(ntest / configs['data']['batch_size'])
print('> Testing model on', ntest, 'data rows with', steps_test, 'steps')

model = lstm.load_network('data/model_saved.h5')

predictions = model.predict_generator(
    generator_strip_xy(data_gen_test, true_values),
    steps=steps_test
)

#Save our predictions
with h5py.File(configs['model']['filename_predictions'], 'w') as hf:
    dset_p = hf.create_dataset('predictions', data=predictions)
    dset_y = hf.create_dataset('true_values', data=true_values)
    
plot_results(predictions[:800], true_values[:800])

Then I got the correct results.
Good luck!