By EmJ


2020-03-26 00:00:19 8 Comments

I have a binary classification problem. I use the following keras model to do my classification.

input1 = Input(shape=(25,6))
x1 = LSTM(200)(input1)
input2 = Input(shape=(24,6))
x2 = LSTM(200)(input2)
input3 = Input(shape=(21,6))
x3 = LSTM(200)(input3)
input4 = Input(shape=(20,6))
x4 = LSTM(200)(input4)
x = concatenate([x1,x2,x3,x4])
x = Dropout(0.2)(x)
x = Dense(200)(x)
x = Dropout(0.2)(x)
output = Dense(1, activation='sigmoid')(x)

However, the results I get is extremely bad. I thought the reason is that I have too many features, thus, needs have more improved layers after the concatenate.
I was also thinking if it would be helpful to used a flatten() layer after the concatenate.
anyway, since I am new to deep learning, I am not so sure how to make this a better model.

I am happy to provide more details if needed.

2 comments

@Ronakrit W. 2020-03-26 03:24:01

Here is what I can suggest

  1. Remove every things that prevent overfitting, such as Dropout and regularizer. What can happen is that your model may not be able to capture the complexity of your data using given layer, so you need to make sure that your model is able to overfit first before adding regularizer.

  2. Now try increase number of Dense layer and number of neuron in each layer until you can see some improvement. There is also a possibility that your data is too noisy or you have only few data to train the model so you can't even produce a useful predictions.

  3. Now if you are LUCKY and you can see overfitting, you can add Dropout and regularizer.

Because every neural network is a gradient base algorithm, you may end up at local minimum. You may also need to run the algorithm multiple times with different initial weight before you can get a good result or You can change your loss function so that you have a convex problem where local minimum is global minimum.

If you can't achieve better result

You may need to try different topology because LSTM is just trying to model a system that assume to have Markov property. you can look at nested-LSTM or something like that, which model the system in the way that next time step is not just depend on current time step.

@EmJ 2020-03-28 01:13:49

Hi, just in case if you have time, please kindly let me know your thoughts on this question: stackoverflow.com/questions/60895974/… :)

@danielcahall 2020-03-26 00:28:27

The Dropout right before the output layer could be problematic. I would suggest removing both Dropout layers and evaluating performance, then re-introduce regularization once the model is performing well on the the training set.

@EmJ 2020-03-26 00:34:42

Thanks a lot for the answer. what is meant by regularisation? Is it possible to show me how to do regularisation for my model in python?

@danielcahall 2020-03-26 00:51:59

Sorry - regularization is a method of incorporating penalty terms in your model and helps avoid overfitting. To clarify, dropout is a form of regularization. I meant that you can re-introduce dropout once the model is performing well. But generally, if you wanted to introduce other forms of regularization such as L2 or L1, it's fairly easy in Keras - described well in the docs: keras.io/regularizers.

Related Questions

Sponsored Content

25 Answered Questions

[SOLVED] How do I concatenate two lists in Python?

20 Answered Questions

[SOLVED] What are metaclasses in Python?

36 Answered Questions

[SOLVED] How to get the current time in Python

  • 2009-01-06 04:54:23
  • user46646
  • 3222626 View
  • 2784 Score
  • 36 Answer
  • Tags:   python datetime time

25 Answered Questions

[SOLVED] Does Python have a ternary conditional operator?

38 Answered Questions

[SOLVED] How do I check whether a file exists without exceptions?

65 Answered Questions

[SOLVED] Calling an external command from Python

42 Answered Questions

[SOLVED] How do I merge two dictionaries in a single expression?

10 Answered Questions

[SOLVED] Does Python have a string 'contains' substring method?

Sponsored Content