By Phylliade


2017-07-20 14:43:10 8 Comments

A Keras model can used as a Tensorflow function on a Tensor, through the functional API, as described here.

So we can do:

from keras.layers import InputLayer

a = tf.placeholder(dtype=tf.float32, shape=(None, 784))

model = Sequential()
model.add(InputLayer(input_tensor=a, input_shape=(None, 784)))
model.add(Dense(32, activation='relu'))
model.add(Dense(10, activation='softmax'))

output = model.output

Which is a tensor:

<tf.Tensor 'dense_24/Softmax:0' shape=(?, 10) dtype=float32>

But, this also works without any InputLayer:

a = tf.placeholder(dtype=tf.float32, shape=(None, 784))

model = Sequential()
model.add(Dense(32, activation='relu', input_shape=(784,)))
model.add(Dense(10, activation='softmax'))

output = model(a)

works, and output has the same shape as before:

<tf.Tensor 'sequential_9/dense_22/Softmax:0' shape=(?, 10) dtype=float32>

I assume the first form permits:

  • to explicitely attach the inputs and outputs as attributes of the model (of the same names), so we can reuse them elsewhere. For example with other TF ops.
  • to transform the tensors given as inputs into Keras inputs, with additional metadata (such as _keras_history as stated in the source code).

But this is not something we cannot do with the second form, so, is there a special usage of the InputLayer (and Input a fortiori) (except for multiple inputs)?
Moreover, the InputLayer is tricky because it's using input_shape differently from other keras layers: we specify the batch size (None here), which is not usually the case...

1 comments

@Jonas Adler 2017-07-22 22:40:48

It would seem that InputLayer has some uses:

  • First, it allows you to give pure tensorflow tensors as is, without specifying their shape. E.g. you could have written

    model.add(InputLayer(input_tensor=a))
    

    This is nice for several obvious reasons, among others less duplication.

  • Second, they allow you to write non-sequential networks with a single input, e.g.

          a
         / \
        /   \
       /     \
    conv1   conv2
      |       |
    

    Without InputLayer you would need to explicitly feed conv1 and conv2 the same tensor, or create an arbitrary identity layer on top of the model. Neither is quite pleasing.

  • Finally, they remove the arbitrary distinction between "layers that are also inputs" and "normal layers". If you use InputLayer you can write code where there is a clear distinction between what layer is the input and what layer does something. This improves code readability and makes refactoring much easier. For example, replacing the first layer becomes just as easy as replacing any other layer, you don't need to think about input_shape.

Related Questions

Sponsored Content

2 Answered Questions

Combine keras functional api with tensorflow

  • 2018-06-01 22:19:35
  • lab_rat
  • 722 View
  • 1 Score
  • 2 Answer
  • Tags:   tensorflow keras

2 Answered Questions

1 Answered Questions

[SOLVED] Keras Sequential model input layer

2 Answered Questions

1 Answered Questions

5 Answered Questions

[SOLVED] Tensor is not an element of this graph; deploying Keras model

1 Answered Questions

[SOLVED] Keras Tensorflow 'Cannot apply softmax to a tensor that is 1D'

  • 2019-04-07 22:12:02
  • Jose Villalta
  • 227 View
  • 0 Score
  • 1 Answer
  • Tags:   tensorflow keras

1 Answered Questions

1 Answered Questions

[SOLVED] Concatening Attention layer with decoder input seq2seq model on Keras

1 Answered Questions

[SOLVED] TensorFlow to Keras Tensor

Sponsored Content