Tf.print() Vs Python Print Vs Tensor.eval()

It seems that in Tensorflow, there are at least three methods to print out the value of a tensor. I've been reading here and there, yet still very confused. These authors seem to

Solution 1:

The native Python print() statement will be called when the graph is build the first time. Check this out:

a = tf.placeholder(shape=None, dtype=tf.int32)
b = tf.placeholder(shape=None, dtype=tf.int32)
print("a is ",a," while b is ",b)
c = tf.add(a, b)
with tf.Session() as sess:
    print(sess.run(c, feed_dict={a: 1, b: 2}))
    print(sess.run(c, feed_dict={a: 3, b: 1}))

By exectuing this code block, the output is:

# a is  Tensor("Placeholder:0", dtype=int32)  while b is  Tensor("Placeholder_1:0", dtype=int32)
# 3
# 4

On the other hand, let us see tf.print():

a = tf.placeholder(shape=None, dtype=tf.int32)
b = tf.placeholder(shape=None, dtype=tf.int32)
print_op = tf.print("a is ",a," while b is ",b)
with tf.control_dependencies([print_op]):
    c = tf.add(a, b)

with tf.Session() as sess:
    print(sess.run(c, feed_dict={a: 1, b: 2}))
    print(sess.run(c, feed_dict={a: 3, b: 1}))

So, according to the output below, we can see that if we add the dependency that the tf.print op must be run whenever c is run, we get to see the output we want to:

# a is  1  while b is  2
# 3
# a is  3  while b is  1
# 4

Finally, tensor.eval() is identical to sess.run(tensor). However, the limitation of tensor.eval() is that you can run it to evaluate a single tensor, while the tf.Session can be used to evaluate multiple tensors sess.run([tensor1, tensor2]). If you ask me, I would always use sess.run(list_of_tensors), to evaluate as many tensors as I want to, and print out their values.

Solution 2:

1. No. The Python print is not called when you call sess.run() later.
   If you want to print when you call sess.run() then you can use tf.print.

2. To print out multiple tensor values in a graph, you should use sess.run() after opening tf.Session(). Sample code is below.

t = tf.constant(42.0)
u = tf.constant(37.0)
pt = tf.print(t)
pu = tf.print(u)
with sess.as_default():
   sess.run([pt, pu])

42
37

3. This answer and this in another question will be helpful.
   tensor.eval() evaluates tensor operation and is not an operator.
   tf.print() is just an operator that prints out given tensor. So after invoking tf.Session(), tf.print() is to be one of the graph nodes.