Keras: Understanding The Number Of Trainable LSTM Parameters

I have run a Keras LSTM demo containing the following code (after line 166): m = 1 model=Sequential() dim_in = m dim_out = m nb_units = 10 model.add(LSTM(input_shape=(None, dim_in

Solution 1:

The params formula holds for the whole layer, not per Keras unit.

Quoting this answer:

[In Keras], the unit means the dimension of the inner cells in LSTM.

LSTM in Keras only define exactly one LSTM block, whose cells is of unit-length.

Directly setting output_size = 10 (like in this comment) correctly yields the 480 parameters.

Solution 2:

Your error lies in the interpretation of terms given on your quoted page (which is admittedly misleading). So n in the reference corresponds to your nb_units, which can be appreciated by the fact that this variable enters quadratically into the given formula, and thus corresponds to the recurrent connectivity, which plays out between the nb_units LSTM cells only.

So, setting output_size = n = 10 in your formula above for params will give the desired 480 parameters.