only send progress bars to stderr if it's a tty

Well, I think there is a choice between tradeoffs.

Its very hard for scripting to deal with progress bars and other such
'not the requested content' artifacts.

CLI programs get precisely two file descriptors to write to.

Suggestions about how to improve the situation without making life
harder for interactive or scripting users will be appreciated - but this
isn't a casual decision that we took.

 status incomplete

Robert Collins wrote:
> Well, I think there is a choice between tradeoffs.
>
> Its very hard for scripting to deal with progress bars and other such
> 'not the requested content' artifacts.
>
> CLI programs get precisely two file descriptors to write to.
>
That is not true. Two output handles are by default provided that are
(in the absence of redirection on the command line) connected to the
terminal.
Any program can use others, and bash even provides a mechanism for
redirecting them (e.g. 3>myfile).

The big issue (TM) is that output to STDERR should be limited to
important things the user must be informed about.
Cluttering that output with non-essential "noise" reduces the value of
that output channel significantly.

> Suggestions about how to improve the situation without making life
> harder for interactive or scripting users will be appreciated - but this
> isn't a casual decision that we took.
>
>
One realistic suggestion is to not output that progress bar at all
unless the user explicitly ASKS for it (presently due to bug 339385
and/or the lack of a config file option this is not possible). One could
also provide a config option to say which output channel to use (
default STDOUT).

Another improvement is to not output progress information if the POSIX
function isatty returns false. Look at what happens when the output was
redirected to a file (this is exactly what the file contains):

[/ ] Build phase 0/2

[- ] Build phase:Building tree 0/1037

[\ ] Build phase:Adding file contents 33/1037

[##| ] Build phase:Adding file contents 320/1037

[#####/ ] Build phase:Adding file contents 625/1037

[######- ] Build phase:Adding file contents 769/1037

[########\ ] Build phase:Adding file contents 938/1037

[#########| ] Build phase 1/2

[#########/ ] Build phase:Apply phase 0/2

[#########- ] Build phase:Apply phase:removing file 0/1

[##############\ ] Build phase:Apply phase 1/2

[##############| ] Build phase:Apply phase:adding file 0/16

bzr uses stdout and stderr consistently for some bzr specific value of consistency.

So, progress reporting will continue to be done on stderr.

Yet, it is very well known that in some contexts (mostly scripting or other usages out of user view), such progress has no value and should not be output at all.

This is worked on and would certainly ends up being controlled by an environment variable.

Automatic detection has limitations I had already run into, disabling and even sometimes choosing the right kind of output can't always be guessed reliably and should be left under user control.

See bug 388275 for the wish to configure this in a file.

bzr does check isatty at the moment, but it checks it only on stdin, whereas it would be more appropriate to check whether stderr is a tty before showing the progress bar.

David, in the case where bzr is showing a progress bar, leaving aside for the moment how we decide when that should be done, where do you think it should be sent, if not to stderr? stdout? /dev/tty?

Martin Pool wrote:
> See bug 388275 for the wish to configure this in a file.
>
> bzr does check isatty at the moment, but it checks it only on stdin,
> whereas it would be more appropriate to check whether stderr is a tty
> before showing the progress bar.
>
> David, in the case where bzr is showing a progress bar, leaving aside
> for the moment how we decide when that should be done, where do you
> think it should be sent, if not to stderr? stdout? /dev/tty?
>
> ** Summary changed:
>
> - progress report should be user configurable
> + don't want progress bars sent to stderr
>
I should have checked all the mail before I replied to the first one!

Despite Matthew's attempts I still stand by the principle that STDERR
should be clean, however I feel even more that stdout must be clean.
The original design in UNIX with stdin,stdout and stderr always being
opened for every program was at the time inspired. What wasn't known
then was the importance of interactive users and their desire to be
reassured with progress bars! In retrospect probably there should have
been another handle for "occasional non-crucial informative output".
While bash does provide a means to use other handles this is not easy to
use and may not be available in other OS's.

As I said in the response to the forum, /dev/tty is not available to
detached scripts or daemons - which also can't benefit from the progress
bar anyway. I do recognise that the least evil place is stderr.

You could avoid the question by providing a config option for the user
to select where to send the progress bar, but that may be gilding the lily.

If the environment-variable/config-file option is available the issue
becomes less important - especially if the default is OFF ;)
The issue can be further reduced by only showing progress when things
happen slowly enough that the user could need reassuring. A progress bar
that changes (and then disappears) so quickly that it cannot be read is
not worth much.

Thanks for taking my remarks seriously,

David.

There is one thing in the current code that's clearly incorrect, and that is that will still send progress to stderr even if it's redirected to a file, which is fairly clearly wrong. (We only check isatty on stdin.) The attached branch fixes this.

This bug has shifted a bit from David's original filing. Here's the rationale:

- He originally asked for progress bars not to be sent to stderr. If we are going to show progress bars, there is no better place to send them.
- He also asked for progress bars to be off by default; I'm not going to do that but there is another bug 388275 asking for a way to permanently turn them off in a file. You can turn them off in current trunk using BZR_PROGRESS_BAR=none.
- He's correct that we don't handle some cases of redirection correctly.

Therefore the remainder of this bug is that autodetection on redirected stderr is imperfect. The attached branch addresses this but has some failing tests.

Martin Pool wrote:
> This bug has shifted a bit from David's original filing. Here's the
> rationale:
>
> - He originally asked for progress bars not to be sent to stderr. If we are going to show progress bars, there is no better place to send them.
>
I recognise and accept that, but still stand by my point that
non-critical output should _only_ go to stderr _if_invited_ to do so....
> - He also asked for progress bars to be off by default; I'm not going to do that but there is another bug 388275 asking for a way to permanently turn them off in a file. You can turn them off in current trunk using BZR_PROGRESS_BAR=none.
>
... therefore they should be off be default.
> - He's correct that we don't handle some cases of redirection correctly.
>
>
Nice to know I got something right ;^)

bug 395998 asks for them to be suppressed for short operations, which was also mentioned in the thread.