By robert


2014-10-09 01:08:28 8 Comments

I have a python script that looks something like this:

for item in collection:
    print "what up"
    #do complicated stuff that takes a long time.

In bash, I run this script by doing the following:

$ python my.py | tee my_file.txt

However, all I see in bash is a blank line until the program finishes. Then, all of the print statements come all at one.

Is this the expected operation of tee? Can I use tee to see the output in real-time?

1 comments

@Robᵩ 2014-10-09 16:50:03

Python, like many programs, tries to minimize the number of times it calls the write system call. It does this by collecting the output of several print statements before it actually writes them to its standard output file. This process is called buffering the output.

When Python is connected to a terminal, it doesn't buffer its output. This makes sense, because the human at the terminal wants to see the output right away.

When Python is writing to a file (or a pipe), it does buffer its output. This also makes sense, because no one will see the output until the process is complete

You can defeat this optimization by calling sys.stdout.flush() whenever you want to force Python to write its buffered output to its standard output file.

In your case, try this:

import sys
...
for item in collection:
    print "what up"
    sys.stdout.flush()
    #do complicated stuff that takes a long time.

@BiBi 2018-01-27 18:57:19

Great explanation! I would add that calling the script with the flag -u would also solve your problem, e.g. python -u my.py | tee my_file.txt

Related Questions

Sponsored Content

44 Answered Questions

[SOLVED] Replacements for switch statement in Python?

11 Answered Questions

[SOLVED] Getting the last element of a list

  • 2009-05-30 19:28:53
  • Janusz
  • 1835945 View
  • 1865 Score
  • 11 Answer
  • Tags:   python list indexing

7 Answered Questions

[SOLVED] How do I get the number of elements in a list?

  • 2009-11-11 00:30:54
  • y2k
  • 3153789 View
  • 1838 Score
  • 7 Answer
  • Tags:   python list

11 Answered Questions

[SOLVED] How to remove a key from a Python dictionary?

36 Answered Questions

[SOLVED] How to get the current time in Python

  • 2009-01-06 04:54:23
  • user46646
  • 3059284 View
  • 2609 Score
  • 36 Answer
  • Tags:   python datetime time

19 Answered Questions

[SOLVED] How to remove an element from a list by index?

  • 2009-03-09 18:16:11
  • Joan Venge
  • 2392273 View
  • 1364 Score
  • 19 Answer
  • Tags:   python list

14 Answered Questions

[SOLVED] How to print to stderr in Python?

10 Answered Questions

6 Answered Questions

[SOLVED] How to replicate tee behavior in Python when using subprocess?

5 Answered Questions

[SOLVED] Force line-buffering of stdout when piping to tee

Sponsored Content