GridSearchCV/RandomizedSearchCV With LSTM

I am stuck on the trying to tune hyperparameters for LSTM via RandomizedSearchCV. My code is below: X_train = X_train.reshape((X_train.shape[0], 1, X_train.shape[1])) X_test = X_te

Solution 1:

This problem is not due to scikit-learn. RandomizedSearchCV does not check the shape of input. That is the work of the individual Transformer or Estimator to establish that the passed input is of correct shape. As you can see from the stack trace, that error is created by imblearn because SMOTE requires data to be 2-D to work.

To avoid that, you can reshape the data manually after SMOTE and before passing it to the LSTM. There are multiple ways to achieve this.

1) You pass 2-D data (without explicitly reshaping as you are doing currently in the following lines):

X_train = X_train.reshape((X_train.shape[0], 1, X_train.shape[1]))
X_test = X_test.reshape((X_test.shape[0], 1, X_test.shape[1]))

to your pipeline and after the SMOTE step, before your clf, reshape the data into 3-D and then pass it to clf.

2) You pass your current 3-D data to the pipeline, transform it into 2-D to be used with SMOTE. SMOTE will then output new oversampled 2-D data which you then again reshape into 3-D.

I think the better option will be 1. Even in that, you can either:

• use your custom class to transform the data from 2-D to 3-D like the following:

  pipe = Pipeline([
      ('oversample', SMOTE(random_state=12)),
  
      # Check out custom scikit-learn transformers
      # You need to impletent your reshape logic in "transform()" method
      ('reshaper', CustomReshaper(),
      ('clf', clf)
  ])
  

• or use the already available Reshape class. I am using Reshape.

So the modifier code would be (See the comments):

# Remove the following two lines, so the data is 2-D while going to "RandomizedSearchCV". 

#    X_train = X_train.reshape((X_train.shape[0], 1, X_train.shape[1]))
#    X_test = X_test.reshape((X_test.shape[0], 1, X_test.shape[1]))


from keras.layers import Reshape

def create_model(activation_1='relu', activation_2='relu',
                 neurons_input = 1, neurons_hidden_1=1,
                 optimizer='Adam' ,):

  model = Sequential()

  # Add this before LSTM. The tuple denotes the last two dimensions of input
  model.add(Reshape((1, X_train.shape[1])))
  model.add(LSTM(neurons_input, 
                 activation=activation_1, 

                 # Since the data is 2-D, the following needs to be changed from "X_train.shape[1], X_train.shape[2]"
                 input_shape=(1, X_train.shape[1]),
                 kernel_initializer=RandomNormal(mean=0.0, stddev=0.05, seed=42), 
                 bias_initializer=RandomNormal(mean=0.0, stddev=0.05, seed=42)))

  model.add(Dense(2, activation='sigmoid'))

  model.compile (loss = 'sparse_categorical_crossentropy', optimizer=optimizer)
  return model