The Bcc Issue

[Philip Hazel]

The Bcc Issue: posted to the exim-users, exim-dev, and ietf-822 mailing lists
-----------------------------------------------------------------------------

The issue of who handles Bcc: header lines has again been raised, and I
am seeking opinions as widely as I can. The requirement is
straightforward: non-Bcc recipients of a message should not see the
addresses of any Bcc recipients, that is, their copies of the message
should not contain Bcc: header lines.

The dispute is over who achieves this end by removing Bcc: lines when
they should not be present. Is it the MUA or the MTA? This is what I
know about current behaviour:

1. Exim (which I maintain) removes Bcc: lines if, and only if, it is
called with the -t option. In that case, it is constructing an
envelope from the header data, and IMHO in doing so it is fulfilling
an MUA function. In all other cases it leaves Bcc: lines alone.

2. It has been reported to me that OpenWave InterMail, MS SMTP, MS
Exchange, and qmail behave in the same way as Exim for incoming SMTP
messages (i.e. they leave Bcc: lines alone).

3. I have also been told that Sendmail, at least in some configurations,
always removes any Bcc: lines from any message that it handles. The
same is apparently true of Postfix and Lotus Notes 5.

4. Among the MUAs, I think that Pine never sends Bcc: lines, but Mutt by
default does, assuming that the MTA will deal with them (Mutt does
not use the -t option).

The (re-)discussion of this issue arose because the combination of
(default configured) Mutt and Exim does not behave the way people
expect. Mutt sends Bcc: lines, and Exim lets them through to all
recipients.

The RFC permits several different behaviours in connection with Bcc
recipients. For copies of the message that are sent to Bcc recipients,
the Bcc: line may be absent, or contain just the address of the single
recipient, or contain the addresses of all the Bcc recipients. For
copies sent to non-Bcc recipients, there should be no Bcc: header line.

If the MUA handles this, it is possible for it to allow any of the
alternative behaviours, as the sender of the message wishes. An MTA has
no means of distinguishing what the sender wanted. If it is to handle
the Bcc: header at all, the only choice is always to remove it. Thus, an
MTA that always removes Bcc: header lines will frustrate the wishes of
someone who wants them included in copies to Bcc recipients. (I found
one posting on the www that complained of exactly this problem.)

Of course, it is possible for MTAs to have configuration options to
change their behaviour. (It is easy to configure Exim always to remove
Bcc: lines, for instance.) However, only clueful sysadmins who are aware
of the issue will investigate such options. Thus, it is important for
the default behaviour to be "correct" in the sense that it is what the
RFCs and the community experts generally agree on.

So .... please vent your opinions and I will try to pull together a
summary in due course.

I am posting this separately to the three list mentioned above because
the Exim lists are subscriber-posting only, and a cross-post might cause
problems with replies.

Philip

--
Philip Hazel University of Cambridge Computing Service,
***@cus.cam.ac.uk Cambridge, England. Phone: [phone].

[Keith Moore]

> The issue of who handles Bcc: header lines has again been raised, and
> I am seeking opinions as widely as I can. The requirement is
> straightforward: non-Bcc recipients of a message should not see the
> addresses of any Bcc recipients, that is, their copies of the message
> should not contain Bcc: header lines.
>
> The dispute is over who achieves this end by removing Bcc: lines when
> they should not be present. Is it the MUA or the MTA?

It's the MUA. MTAs should not be altering message headers other than
to add Received fields and a Return-Path field on delivery.

I believe the confusion results from the common practice of asking
"sendmail -t" (be it the original sendmail program or some clone) to
perform some of the MUA's functions - in particular, generating
envelope recipient lists from the to/cc/bcc message header fields.
In this case "sendmail" is still performing an MUA function, even
though we think of it as an MTA. (it might be the case that sendmail
is older than the MHS model and the formal MTA/MUA split)

So when an MUA asks a "sendmail -t" interface to parse header fields
generate a recipient list, and submit the message, that sendmail-like
program should be removing Bcc fields from the copy of a message sent to
any recipient that isn't in the Bcc list. It MAY remove the Bcc field
from all copies and still comply with the specifications, but this is
Bad.

OTOH, when an MUA uses SMTP (or the SUBMISSION protocol) to submit a
message for delivery, it's the MUA's responsibility to generate multiple
envelopes for any message containing Bcc's - one envelope for the Bcc
recipients (which may include the Bcc header field) and one envelope
for the non-Bcc'ed recipients (which MUST NOT include the Bcc field).
The SMTP or SUBMISSION server should not be removing or interpreting
the Bcc fields in this case.

The MUA is never required to include a Bcc field in the messages sent
to Bcc recipients - but it's the Right Thing to do.

-Keith

[Nathaniel Borenstein]

As one who has always despised the whole concept of bcc's, everything I
say should be taken with a large grain of salt. However...

The essence of a "bcc" is "I want a copy to go to this person, but I
don't want the 'real' recipient to know it." Given that
motivation/intent, writing it down in a header field is a somewhat
dubious concept to begin with. Anyone who puts in a "bcc" field
pretty obviously intends for it to be taken out before it reaches the
ultimate recipient. This suggests, however, an obvious way to handle
such headers: Take them out of the headers and (probably) inject them
into the SMTP recipient stream as soon as you possibly can!

In other words, a conservative default would be that *any* software
that received a message with such a field should feel very free to take
out the field, and should probably send the bcc as well. The nastiest
failure case is when the message is delivered with the bcc field
intact, so it seems to me we should try to err in the other direction
and pre-empt a faux pas if we can.

Thus, I would argue ever-so-impartially that mutt and exim are both
broken and should both be fixed. Exim might be easier to fix, since
it's just a matter of changing the default setting. Mutt's objections
might be more philosophical (see my PS) but physicians need to treat
patients who haven't followed their advice, and Mutt should do its best
to protect its users if they send bcc's. -- Nathaniel

PS -- Aside from the ambiguities in their
interpretation/implementation, I despise bcc's as Temptations to Sin.
While a few situations (e.g. fyis to celebrities) clearly justify
bcc's, more often bcc's promote poor communication and bad behavior.
The Puritan in me thinks you should have to type "~b" at an absolute
minimum in order to send a bcc, and probably something even more
obscure. :-) But it's still the software's job to try to minimize the
harm that happens to you once you use the feature, which is why I
argue for implementing this at more or less *every* protocol hop. --
NB

On Aug 13, 2004, at 6:24 AM, Philip Hazel wrote:

>
> The Bcc Issue: posted to the exim-users, exim-dev, and ietf-822
> mailing lists
> -----------------------------------------------------------------------
> ------
>
> The issue of who handles Bcc: header lines has again been raised, and I
> am seeking opinions as widely as I can. The requirement is
> straightforward: non-Bcc recipients of a message should not see the
> addresses of any Bcc recipients, that is, their copies of the message
> should not contain Bcc: header lines.
>
> The dispute is over who achieves this end by removing Bcc: lines when
> they should not be present. Is it the MUA or the MTA? This is what I
> know about current behaviour:
>
> 1. Exim (which I maintain) removes Bcc: lines if, and only if, it is
> called with the -t option. In that case, it is constructing an
> envelope from the header data, and IMHO in doing so it is fulfilling
> an MUA function. In all other cases it leaves Bcc: lines alone.
>
> 2. It has been reported to me that OpenWave InterMail, MS SMTP, MS
> Exchange, and qmail behave in the same way as Exim for incoming SMTP
> messages (i.e. they leave Bcc: lines alone).
>
> 3. I have also been told that Sendmail, at least in some
> configurations,
> always removes any Bcc: lines from any message that it handles. The
> same is apparently true of Postfix and Lotus Notes 5.
>
> 4. Among the MUAs, I think that Pine never sends Bcc: lines, but Mutt
> by
> default does, assuming that the MTA will deal with them (Mutt does
> not use the -t option).
>
> The (re-)discussion of this issue arose because the combination of
> (default configured) Mutt and Exim does not behave the way people
> expect. Mutt sends Bcc: lines, and Exim lets them through to all
> recipients.
>
> The RFC permits several different behaviours in connection with Bcc
> recipients. For copies of the message that are sent to Bcc recipients,
> the Bcc: line may be absent, or contain just the address of the single
> recipient, or contain the addresses of all the Bcc recipients. For
> copies sent to non-Bcc recipients, there should be no Bcc: header line.
>
> If the MUA handles this, it is possible for it to allow any of the
> alternative behaviours, as the sender of the message wishes. An MTA has
> no means of distinguishing what the sender wanted. If it is to handle
> the Bcc: header at all, the only choice is always to remove it. Thus,
> an
> MTA that always removes Bcc: header lines will frustrate the wishes of
> someone who wants them included in copies to Bcc recipients. (I found
> one posting on the www that complained of exactly this problem.)
>
> Of course, it is possible for MTAs to have configuration options to
> change their behaviour. (It is easy to configure Exim always to remove
> Bcc: lines, for instance.) However, only clueful sysadmins who are
> aware
> of the issue will investigate such options. Thus, it is important for
> the default behaviour to be "correct" in the sense that it is what the
> RFCs and the community experts generally agree on.
>
> So .... please vent your opinions and I will try to pull together a
> summary in due course.
>
> I am posting this separately to the three list mentioned above because
> the Exim lists are subscriber-posting only, and a cross-post might
> cause
> problems with replies.
>
> Philip
>
> --
> Philip Hazel University of Cambridge Computing Service,
> ***@cus.cam.ac.uk Cambridge, England. Phone: [phone].
>
>
>

[ned+]

> As one who has always despised the whole concept of bcc's, everything I
> say should be taken with a large grain of salt. However...

> The essence of a "bcc" is "I want a copy to go to this person, but I
> don't want the 'real' recipient to know it." Given that
> motivation/intent, writing it down in a header field is a somewhat
> dubious concept to begin with. Anyone who puts in a "bcc" field
> pretty obviously intends for it to be taken out before it reaches the
> ultimate recipient. This suggests, however, an obvious way to handle
> such headers: Take them out of the headers and (probably) inject them
> into the SMTP recipient stream as soon as you possibly can!

This IMO completely misses the point of the bcc field. The reason the
field is there is not to notify all the recipients that a blind copy
was made. That would be silly. Rather, the point is to notify the
bcc recipient that his or her copy is a bcc and should be handled
accordingly.

> In other words, a conservative default would be that *any* software
> that received a message with such a field should feel very free to take
> out the field, and should probably send the bcc as well.

This approach actually presents something of a security exposure, as it results
in someone getting a copy of message without any indication it was done as a
bcc. This is the reason that some UAs have started stuffing "This is a bcc"
text into the message body (often without regard to things like MIME,
unfortunately).

> The nastiest
> failure case is when the message is delivered with the bcc field
> intact, so it seems to me we should try to err in the other direction
> and pre-empt a faux pas if we can.

The nastiest case is when a MUA incorrectly assumes that bcc will
be deleted somewhere down the line and it isn't. But the case where
the MUA assumes that bcc will be carried through and it isn't also
can lead to exposures, albeit less directly.

Ned

[Dave Crocker]

Folks,

I think Nathaniel's note does an excellent job of representing a popular
line of thinking about these issues. It's critical of that line of
thinking, rather than of Nathaniel...

We need to make sure that our assertions about human communications
issues have some relationship to the normal, non-computer range of human
behaviors. Unfortunately in the world of email standards,tend to be both
arbitrary and restrictive in those assertions.


NB> The essence of a "bcc" is "I want a copy to go to this person, but I
NB> don't want the 'real' recipient to know it." Given that

Nit pick: The usual term is 'primary', rather than 'real'.


NB> motivation/intent, writing it down in a header field is a somewhat
NB> dubious concept to begin with.

The concept of making such a note for the author's agent, such as a
secretary, is not dubious at all. It is the only way the author can
tell the agent how to handle the message.


NB> Anyone who puts in a "bcc" field
NB> pretty obviously intends for it to be taken out before it reaches the
NB> ultimate recipient.

In fact that did not used to be standard practise.

The benefit of having a BCC field in a message is that the recipient
does not need to guess why they got the message. (Lest anyone says 'if i
got the message but am not listed in it then i am obviously a bcc
recipient' we need to observe both the baroqueness (baroquity?) of such
a (non-)labeling technique and its potential to misbehave.)

In fact it is not unreasonable to have each bcc recipient get a copy of
the message with a bcc header that contains (only) their name in it.

In some scenarios, it would even be reasonable to have all the bcc
recipients get the same copy and have it list all of them. This is a
way to keep a special (closed) group informed about some public
activity, and permit them to continue private discussions about it.


NB> This suggests, however, an obvious way to handle
NB> such headers: Take them out of the headers and (probably) inject them
NB> into the SMTP recipient stream as soon as you possibly can!

NB> In other words, a conservative default would be that *any* software
NB> that received a message with such a field should feel very free to take
NB> out the field, and should probably send the bcc as well.

I am trying to imagine the meta-policy that is behind this being
generally acceptable: any software that receives a message with some
objectionable headers should feel free to take out the field?


NB> The nastiest
NB> failure case is when the message is delivered with the bcc field
NB> intact, so it seems to me we should try to err in the other direction
NB> and pre-empt a faux pas if we can.

First of all, we are talking about the MSA or MUA. Not 'any software'.

Second of all, Internet standards like this would do better to have no
'default' for such situations. Defaults invite problems with the
expectation differing between the creator and the consumer of the
defaulted material.


NB> The Puritan in me thinks you should have to type "~b" at an absolute
NB> minimum in order to send a bcc, and probably something even more
NB> obscure. :-) But it's still the software's job to try to minimize the
NB> harm that happens to you once you use the feature, which is why I
NB> argue for implementing this at more or less *every* protocol hop. --

but, then, we don't do user interface standardization here.

d/
--
Dave Crocker <dcrocker-at-brandenburg-dot-com>
Brandenburg InternetWorking <www.brandenburg.com>
Sunnyvale, CA USA <tel:[phone]>

[Nathaniel Borenstein]

It sure would have been good if we had defined this about 20 years ago.

It was not my intention to imply that you shouldn't leave the bcc
header in for the viewing of the person named in it, merely that it be
removed from the view of anyone *else* as early in the process as
possible. I don't much mind leaving in the bcc that explains why you
received it, although I tend to prefer "Apparently-to" for that purpose
(yes, I know, totally non standard). But the bottom line is that
software uses bcc for two very different functions: to tell another
email transport-related entity to send a bcc, or to tell the user that
he has received a bcc. I don't see how this dual use can possibly be a
good state of affairs.

For me, though, the most important issue is not violating user-level
expectations, particularly when they are privacy-sensitive.
Maintaining the alleged purity of the email architecture is not a
winning argument by comparison. -- Nathaniel



On Aug 13, 2004, at 5:52 PM, Dave Crocker wrote:

>
> Folks,
>
> I think Nathaniel's note does an excellent job of representing a
> popular
> line of thinking about these issues. It's critical of that line of
> thinking, rather than of Nathaniel...
>
> We need to make sure that our assertions about human communications
> issues have some relationship to the normal, non-computer range of
> human
> behaviors. Unfortunately in the world of email standards,tend to be
> both
> arbitrary and restrictive in those assertions.
>
>
> NB> The essence of a "bcc" is "I want a copy to go to this person, but
> I
> NB> don't want the 'real' recipient to know it." Given that
>
> Nit pick: The usual term is 'primary', rather than 'real'.
>
>
> NB> motivation/intent, writing it down in a header field is a somewhat
> NB> dubious concept to begin with.
>
> The concept of making such a note for the author's agent, such as a
> secretary, is not dubious at all. It is the only way the author can
> tell the agent how to handle the message.
>
>
> NB> Anyone who puts in a "bcc" field
> NB> pretty obviously intends for it to be taken out before it reaches
> the
> NB> ultimate recipient.
>
> In fact that did not used to be standard practise.
>
> The benefit of having a BCC field in a message is that the recipient
> does not need to guess why they got the message. (Lest anyone says 'if
> i
> got the message but am not listed in it then i am obviously a bcc
> recipient' we need to observe both the baroqueness (baroquity?) of such
> a (non-)labeling technique and its potential to misbehave.)
>
> In fact it is not unreasonable to have each bcc recipient get a copy of
> the message with a bcc header that contains (only) their name in it.
>
> In some scenarios, it would even be reasonable to have all the bcc
> recipients get the same copy and have it list all of them. This is a
> way to keep a special (closed) group informed about some public
> activity, and permit them to continue private discussions about it.
>
>
> NB> This suggests, however, an obvious way to handle
> NB> such headers: Take them out of the headers and (probably) inject
> them
> NB> into the SMTP recipient stream as soon as you possibly can!
>
> NB> In other words, a conservative default would be that *any* software
> NB> that received a message with such a field should feel very free to
> take
> NB> out the field, and should probably send the bcc as well.
>
> I am trying to imagine the meta-policy that is behind this being
> generally acceptable: any software that receives a message with some
> objectionable headers should feel free to take out the field?
>
>
> NB> The nastiest
> NB> failure case is when the message is delivered with the bcc field
> NB> intact, so it seems to me we should try to err in the other
> direction
> NB> and pre-empt a faux pas if we can.
>
> First of all, we are talking about the MSA or MUA. Not 'any software'.
>
> Second of all, Internet standards like this would do better to have no
> 'default' for such situations. Defaults invite problems with the
> expectation differing between the creator and the consumer of the
> defaulted material.
>
>
> NB> The Puritan in me thinks you should have to type "~b" at an
> absolute
> NB> minimum in order to send a bcc, and probably something even more
> NB> obscure. :-) But it's still the software's job to try to
> minimize the
> NB> harm that happens to you once you use the feature, which is why I
> NB> argue for implementing this at more or less *every* protocol hop.
> --
>
> but, then, we don't do user interface standardization here.
>
> d/
> --
> Dave Crocker <dcrocker-at-brandenburg-dot-com>
> Brandenburg InternetWorking <www.brandenburg.com>
> Sunnyvale, CA USA <tel:[phone]>
>
>
>

[Bruce Lilly]

Nathaniel Borenstein wrote:

> But the bottom line is that software uses bcc
> for two very different functions: to tell another email
> transport-related entity to send a bcc, or to tell the user that he has
> received a bcc.

Not just the (Bcc recipient) user; also to remind the author (via his
file copy) to whom he sent bccs (perhaps some time after the fact).

[Nathaniel Borenstein]

I stand corrected: Three different functions. The line

BCC: ***@guppylake.com

might better be represented as

I-Sent-a-BCC-To: ***@guppylake.com (for my files)
BCC-Should-Be-Sent-To: ***@guppylake.com
(written by a UA as a request to an MTA)
Why-You-Got-This: You were BCC'ed as ***@guppylake.com (for viewing
by the bcc recipient)

None of this makes me any more fond of bcc's. -- N


On Aug 13, 2004, at 7:15 PM, Bruce Lilly wrote:

>
> Nathaniel Borenstein wrote:
>
>> But the bottom line is that software uses bcc
>> for two very different functions: to tell another email
>> transport-related entity to send a bcc, or to tell the user that he
>> has
>> received a bcc.
>
> Not just the (Bcc recipient) user; also to remind the author (via his
> file copy) to whom he sent bccs (perhaps some time after the fact).
>
>
>

[Tony Hansen]

Nathaniel Borenstein wrote:

>
> I stand corrected: Three different functions. The line
>
> BCC: ***@guppylake.com
>
> might better be represented as
>
> I-Sent-a-BCC-To: ***@guppylake.com (for my files)
> BCC-Should-Be-Sent-To: ***@guppylake.com
> (written by a UA as a request to an MTA)
> Why-You-Got-This: You were BCC'ed as ***@guppylake.com (for
> viewing by the bcc recipient)

Nit here on your second item: An MTA never looks at the header of a
message! The MTA is expected to look at an envelope, not the header. If
you're passing a message to something that IS expected to look at the
To/Cc/Bcc headers and act on them, that thing is NOT an MTA.

Even MSA's are not expected to look at the To/Cc/Bcc headers.

Now, if you want to argue that some software DOES look at those headers
to determine the addresses to send a message to, that's fine. Those
software packages are working as part of the MUA system, and it's a
private contract between those software packages to do so.

But an MTA should NOT be looking at the To/Cc/Bcc headers.

Tony Hansen
***@att.com

[Nathaniel Borenstein]

OK, fine, got me again, I meant an MSA -- which is generally, in
practice, the same code as an MTA, but functioning as an MSA (e.g.
sendmail listening on port 25 or 587 for message submission).

As to why this is worse than the similar situation with To/CC/etc
(Keith's question) -- nobody ever *intends* for those field values to
be hidden from the recipient, as they do with bcc. -- Nathaniel

On Aug 13, 2004, at 10:03 PM, Tony Hansen wrote:

> Nathaniel Borenstein wrote:
>
>> I stand corrected: Three different functions. The line
>> BCC: ***@guppylake.com
>> might better be represented as
>> I-Sent-a-BCC-To: ***@guppylake.com (for my files)
>> BCC-Should-Be-Sent-To: ***@guppylake.com
>> (written by a UA as a request to an MTA)
>> Why-You-Got-This: You were BCC'ed as ***@guppylake.com
>> (for viewing by the bcc recipient)
>
> Nit here on your second item: An MTA never looks at the header of a
> message! The MTA is expected to look at an envelope, not the header.
> If you're passing a message to something that IS expected to look at
> the To/Cc/Bcc headers and act on them, that thing is NOT an MTA.
>
> Even MSA's are not expected to look at the To/Cc/Bcc headers.
>
> Now, if you want to argue that some software DOES look at those
> headers to determine the addresses to send a message to, that's fine.
> Those software packages are working as part of the MUA system, and
> it's a private contract between those software packages to do so.
>
> But an MTA should NOT be looking at the To/Cc/Bcc headers.
>
> Tony Hansen
> ***@att.com
>
>
>
>

[Tony Hansen]

An MSA listening on either port 25 or 587 should NOT be looking at
To/Cc/Bcc headers. They should only be looking at RCPT TO. See RFC 2476.

Tony Hansen
***@att.com

Nathaniel Borenstein wrote:

> OK, fine, got me again, I meant an MSA -- which is generally, in
> practice, the same code as an MTA, but functioning as an MSA (e.g.
> sendmail listening on port 25 or 587 for message submission).
>
> As to why this is worse than the similar situation with To/CC/etc
> (Keith's question) -- nobody ever *intends* for those field values to be
> hidden from the recipient, as they do with bcc. -- Nathaniel

[ned+]

> An MSA listening on either port 25 or 587 should NOT be looking at
> To/Cc/Bcc headers. They should only be looking at RCPT TO. See RFC 2476.

Or flipping it around, an agent that examines To/CC/Bcc headers around the time
of message submission is considered to be part of the MUA. The fact that
this can happen in a different process or even on a different machine
doesn't make it an MSA.

Ned

[Keith Moore]

>> An MSA listening on either port 25 or 587 should NOT be looking at
>> To/Cc/Bcc headers. They should only be looking at RCPT TO. See RFC
>> 2476.
>
> Or flipping it around, an agent that examines To/CC/Bcc headers around
> the time
> of message submission is considered to be part of the MUA.

perhaps, but an agent should not be performing that part of the MUA's
function for messages accepted via the SUBMISSION protocol.

Keith

[ned+]

> > > An MSA listening on either port 25 or 587 should NOT be looking at
> > > To/Cc/Bcc headers. They should only be looking at RCPT TO. See RFC
> > > 2476.

> > Or flipping it around, an agent that examines To/CC/Bcc headers around
> > the time
> > of message submission is considered to be part of the MUA.

> perhaps, but an agent should not be performing that part of the MUA's
> function for messages accepted via the SUBMISSION protocol.

I never said it was or should do so. But what people actually run on various
ports is, in the final analysis, their own affair. We can only state whether or
not something is standards compliant. Something that performs MUA functions on
server side clearly is not compliant with either SMTP or SUBMISSION.

Ned

[Keith Moore]

>> perhaps, but an agent should not be performing that part of the MUA's
>> function for messages accepted via the SUBMISSION protocol.
>
> I never said it was or should do so. But what people actually run on
> various
> ports is, in the final analysis, their own affair. We can only state
> whether or
> not something is standards compliant.

we can do a bit better than that - we can also state that we design and
test things to interoperate when they comply with the standards, and
not (in general) under other conditions. if someone writes or
configures a protocol engine to deviate from the standards, he does so
at the risk of degrading interoperability, and impairing the network
for his own users and possibly for others. very few users have the
background and analysis skills to understand the implications of
deviating from the standards.

[Keith Moore]

>> we can do a bit better than that - we can also state that we design
>> and test things to interoperate when they comply with the standards,
>> and not (in general) under other conditions. if someone writes or
>> configures a protocol engine to deviate from the standards, he does
>> so at the risk of degrading interoperability, and impairing the
>> network for his own users and possibly for others. very few users
>> have the background and analysis skills to understand the
>> implications of deviating from the standards.
>
> And when the standards are as clear as muck, don't we have an
> obligation then?
>
> I can't count the number of communications snafus I've seen that I
> would attribute to the bcc feature. At the very least, we should
> provide a clear model for implementors to follow and convey to users.
> We certainly haven't done that yet, and I'm not sure how we can at
> this point.

I do think we have an obligation to make specifications clear, and I do
think it's possible to improve over what we currently have.

Keith

[Nathaniel Borenstein]

On Aug 16, 2004, at 7:55 AM, Keith Moore wrote:
>
> we can do a bit better than that - we can also state that we design
> and test things to interoperate when they comply with the standards,
> and not (in general) under other conditions. if someone writes or
> configures a protocol engine to deviate from the standards, he does so
> at the risk of degrading interoperability, and impairing the network
> for his own users and possibly for others. very few users have the
> background and analysis skills to understand the implications of
> deviating from the standards.

And when the standards are as clear as muck, don't we have an
obligation then?

I can't count the number of communications snafus I've seen that I
would attribute to the bcc feature. At the very least, we should
provide a clear model for implementors to follow and convey to users.
We certainly haven't done that yet, and I'm not sure how we can at this
point. -- Nathaniel

[Dave Crocker]

Tony,

TH> An MSA listening on either port 25 or 587 should NOT be looking at
TH> To/Cc/Bcc headers. They should only be looking at RCPT TO. See RFC 2476.


>From the mail-arch writing effort, I am not so clear about the boundary
between MUA and MSA in the community's mind.

I'd like to change that.

In general, my sense is that the MSA is entitled to perform quite a bit
of MUA funtionality, in order to off-load the MUA, both computationally
and in terms of network traffic.

If this is a correct assessment, then messing with headers is
acceptable.

I would very much appreciate getting some dialogue about this on the
list.


d/
--
Dave Crocker <dcrocker-at-brandenburg-dot-com>
Brandenburg InternetWorking <www.brandenburg.com>
Sunnyvale, CA USA <tel:[phone]>

[Bruce Lilly]

Dave Crocker wrote:

> In general, my sense is that the MSA is entitled to perform quite a bit
> of MUA funtionality, in order to off-load the MUA, both computationally
> and in terms of network traffic.
>
> If this is a correct assessment, then messing with headers is
> acceptable.
>
> I would very much appreciate getting some dialogue about this on the
> list.

This message serves a couple of purposes. When composed (within Mozilla
1.7), it included a Bcc field. Mozilla is configured to send to sendmail
listening on the submission port [the sendmail process happens to be
running on the same host as Mozilla in this instance]. Mozilla is also
configured to save a copy of sent messages in Mozilla format in a local
"folder". The Bcc copy should be sent by sendmail to a Cyrus IMAP
server (also running on the same host, as it happens) via LMTP. The
list should receive a copy w/o the Bcc field. Mozilla has little in
the way of configuration options for sending -- I'm not sure if it
checks Bccs and sends multiple messages via port 587, or if it relies
on the submission server to do that processing; but I intend to run
a traffic sniffer to find out, and I'll report anything significant
in a subsequent message.

[Bruce Lilly]

Bruce Lilly wrote:

> This message serves a couple of purposes. When composed (within Mozilla
> 1.7), it included a Bcc field. Mozilla is configured to send to sendmail
> listening on the submission port [the sendmail process happens to be
> running on the same host as Mozilla in this instance]. Mozilla is also
> configured to save a copy of sent messages in Mozilla format in a local
> "folder". The Bcc copy should be sent by sendmail to a Cyrus IMAP
> server (also running on the same host, as it happens) via LMTP. The
> list should receive a copy w/o the Bcc field. Mozilla has little in
> the way of configuration options for sending -- I'm not sure if it
> checks Bccs and sends multiple messages via port 587, or if it relies
> on the submission server to do that processing; but I intend to run
> a traffic sniffer to find out, and I'll report anything significant
> in a subsequent message.

Mozilla sent the message (once, with no Bcc header field) to the
MSA, with envelope recipients (RCPT TO) set to all of the recipients
specified, including bcc. The local Mozilla "Sent" "folder" copy
includes the Bcc field. The copy sent to the bcc address (and to the
list -- it was sent only once to the MSA -- does not include any Bcc
header field).

Regarding functionality discussed:
1. I-sent-a-bcc-to only appears in the local Mozilla "folder". It
therefore cannot be accessed (w/o some contortions and hoop-jumping)
anywhere else, or by any other MUA.
2. Bcc-should-be-sent-to is apparently how Mozilla uses the field
3. Why-you-got-this doesn't apply to the bcc recipient -- he has no
indication that the message was sent as a bcc.
4. Non-reply-to seems to be another Mozilla trait; if I go to the
local copy and try "Reply All", there's no copy sent to the bcc
address.

#1 and #3 are apparently disadvantages of the way this particular
combination of software operates. Sending separate Bcc and non-bcc
copies would be preferable functionally, but of course requires
additional bandwidth. The bandwidth issue could theoretically be
handled by offloading the "splitting" functionality on an MTA closer
to the Internet backbone, e.g. using an ESMTP extension as Jacob has
suggested.

[Tony Hansen]

Bruce Lilly wrote:

> Sending separate Bcc and non-bcc copies would be preferable
> functionally, but of course requires additional bandwidth. The
> bandwidth issue could theoretically be handled by offloading the
> "splitting" functionality on an MTA closer to the Internet backbone,
> e.g. using an ESMTP extension as Jacob has suggested.

Given my druthers, I'd much rather see it be an extension only to the
MSA SUBMIT protocol and not one that is expected to be implemented in MTAs.

Tony Hansen
***@att.com

[Bruce Lilly]

Tony Hansen wrote:

> Given my druthers, I'd much rather see it be an extension only to the
> MSA SUBMIT protocol and not one that is expected to be implemented in MTAs.

For the issue of efficient transfer where there is a bottleneck
(e.g. a dialup connection to an ISP), it might not be practical
to restrict it to the SUBMIT protocol; not all ISPs operate
servers on port 587.

[Keith Moore]

> > Given my druthers, I'd much rather see it be an extension only to the
> > MSA SUBMIT protocol and not one that is expected to be implemented in MTAs.
>
> For the issue of efficient transfer where there is a bottleneck
> (e.g. a dialup connection to an ISP), it might not be practical
> to restrict it to the SUBMIT protocol; not all ISPs operate
> servers on port 587.

if they're going to upgrade their servers to support the extension,
they may as well upgrade them to listen on port 587 at the same time.

this really needs to be an extension for SUBMIT only

Keith

[ned+]

> > Given my druthers, I'd much rather see it be an extension only to the
> > MSA SUBMIT protocol and not one that is expected to be implemented in MTAs.

> For the issue of efficient transfer where there is a bottleneck
> (e.g. a dialup connection to an ISP), it might not be practical
> to restrict it to the SUBMIT protocol; not all ISPs operate
> servers on port 587.

First, RFC 2746 is quite clear: Port 25 MAY be used for SUBMIT if for whatever
reason 587 cannot be used.

More generally, what port something is run on is largely irrelevant; the
important issues are service and protocol semantics.

Of course this doesn't mean someone won't attempt to use SUBMIT-specific
extensions in regular SMTP. But just because some clueless ISP engages in
egregiously bad practice is no reason to break our protocol specifications. I
am dubious about the value of such an extension in any case for all the reason
I've previsouly given; I would vehemently oppose making it part of regular
SMTP.

Ned

[Bruce Lilly]

***@mrochek.com wrote:

> First, RFC 2746 is quite clear: Port 25 MAY be used for SUBMIT if for whatever
> reason 587 cannot be used.

2476 -- otherwise OK.

> More generally, what port something is run on is largely irrelevant; the
> important issues are service and protocol semantics.

If the protocols are to be treated as fundamentally different -- and 2476
says that an MSA runs ESMTP with some restrictions (N.B. not extensions) --
then that raises the question of how a client which connects to a server
on port 25 is able to determine whether that server is an MSA or an MTA.

> Of course this doesn't mean someone won't attempt to use SUBMIT-specific
> extensions in regular SMTP.

At the moment, as I understand it, there are no such "extensions", there
are only "restrictions" [RFC 2476 section 3.1].

> But just because some clueless ISP engages in
> egregiously bad practice is no reason to break our protocol specifications. I
> am dubious about the value of such an extension in any case for all the reason
> I've previsouly given; I would vehemently oppose making it part of regular
> SMTP.

It may be the case that presence of one or more hypothetical submission-
specific extensions might serve to distinguish an MSA from an MTA. That
would mean that the presence of such an extension would indicate that the
server in question is an MSA (running ESMTP), not an MTA (running ESMTP).

Of course if a particular MSA doesn't support any of the hypothetical
MSA-specific extensions, there is still the issue of how a client is
supposed to be able to determine whether it has connected to an MTA or
an MSA. What do you suggest?

[n***@mrochek.com]

> ***@mrochek.com wrote:

> > First, RFC 2746 is quite clear: Port 25 MAY be used for SUBMIT if for whatever
> > reason 587 cannot be used.

> 2476 -- otherwise OK.

> > More generally, what port something is run on is largely irrelevant; the
> > important issues are service and protocol semantics.

> If the protocols are to be treated as fundamentally different -- and 2476
> says that an MSA runs ESMTP with some restrictions (N.B. not extensions) --
> then that raises the question of how a client which connects to a server
> on port 25 is able to determine whether that server is an MSA or an MTA.

It would have to be configured to know. But this is for the most part an
entirely academic point, since most clients are only prepared to perform
submission activities. Clients prepared to do relaying, with all it implies,
are hardly the norm.

> > Of course this doesn't mean someone won't attempt to use SUBMIT-specific
> > extensions in regular SMTP.

> At the moment, as I understand it, there are no such "extensions", there
> are only "restrictions" [RFC 2476 section 3.1].

> > But just because some clueless ISP engages in
> > egregiously bad practice is no reason to break our protocol specifications. I
> > am dubious about the value of such an extension in any case for all the reason
> > I've previsouly given; I would vehemently oppose making it part of regular
> > SMTP.

> It may be the case that presence of one or more hypothetical submission-
> specific extensions might serve to distinguish an MSA from an MTA. That
> would mean that the presence of such an extension would indicate that the
> server in question is an MSA (running ESMTP), not an MTA (running ESMTP).

> Of course if a particular MSA doesn't support any of the hypothetical
> MSA-specific extensions, there is still the issue of how a client is
> supposed to be able to determine whether it has connected to an MTA or
> an MSA. What do you suggest?

The idea of offering a specific extension saying "this is a submission server"
or "this is a SMTP relay server" was considered and rejected. I doubt very
much if revisiting this issue at this time will produce a different result. As
such, should the issue of a client needing to know the different ever come up,
the only way to do it would be with a configuration option.

Ned

[Randall Gellens]

At 5:33 PM -0400 8/16/04, Bruce Lilly wrote:

> Tony Hansen wrote:
>
>> Given my druthers, I'd much rather see it be an extension only to the
>> MSA SUBMIT protocol and not one that is expected to be implemented in MTAs.
>
> For the issue of efficient transfer where there is a bottleneck
> (e.g. a dialup connection to an ISP), it might not be practical
> to restrict it to the SUBMIT protocol; not all ISPs operate
> servers on port 587.

It's permitted to run an MSA on port 25; the issue isn't so much
which port but which intended functionality: MTA or MSA. A server
intended to accept connections from authorized users and receive
initial submissions of mail is clearly an MSA. Putting it on port
587 just helps make this more clear, and allows for a clear
separation of security policies between relay and submission. And
has been noted, if you're upgrading anyway, it makes sense to listen
on 587.
--
Randall Gellens
Opinions are personal; facts are suspect; I speak for myself only
-------------- Randomly-selected tag: ---------------
When news of Alexander Graham Bell's invention reached the
United Kingdom, the chief engineer of the British Post Office
failed to be impressed. "The Americans," he said loftily,
have need of the telephone -- but we do not. We have plenty
of messenger boys..."

[Tony Hansen]

Dave Crocker wrote:

> From the mail-arch writing effort, I am not so clear about the boundary
> between MUA and MSA in the community's mind. I'd like to change that.
> In general, my sense is that the MSA is entitled to perform quite a bit
> of MUA funtionality, in order to off-load the MUA, both computationally
> and in terms of network traffic.
> If this is a correct assessment, then messing with headers is
> acceptable.

There are headers and then there are headers. :-)

The current SUBMIT protocol clearly identifies which headers it is
allowed to mess around with. The To/Cc/Bcc headers are not in that list.

But, I do agree with your sentiment -- SUBMIT was clearly intended to
and designed to off-load the MUA from certain tasks. So I could see
there being an extension to the SUBMIT protocol that allowed it to look
at the To/Cc/Bcc headers and acted accordingly. But it would have to be
an extension.

Tony Hansen
***@att.com

[Dave Crocker]

Tony,

TH> The current SUBMIT protocol clearly identifies which headers it is
TH> allowed to mess around with. The To/Cc/Bcc headers are not in that list.

TH> But, I do agree with your sentiment -- SUBMIT was clearly intended to
TH> and designed to off-load the MUA from certain tasks. So I could see

The point raised by Ned(?) -- that an MUA function that is off-loaded to
a server does not make it an MSA function -- is well-taken.

I'm trying to assess people's views on the PRECISE dividing line
between MUA and MSA, rather than trying to espouse a particular view.

I'm fine with the idea of a "split" MUA (and that's a term from the
mid-70's), though it does beg the question of the protocol used between
the local and remote MUA portions.


d/
--
Dave Crocker <dcrocker-at-brandenburg-dot-com>
Brandenburg InternetWorking <www.brandenburg.com>
Sunnyvale, CA USA <tel:[phone]>

[Randall Gellens]

At 4:47 PM -0400 8/16/04, Tony Hansen wrote:

> The current SUBMIT protocol clearly identifies which headers it is
> allowed to mess around with. The To/Cc/Bcc headers are not in that
> list.

RFC 2476 does permit some alteration of address headers:

> 8.7. Resolve Aliases
>
> The MSA MAY resolve aliases (CNAME records) for domain names, in the
> envelope and optionally in address fields of the header, subject to
> local policy.
>
> NOTE: Unconditionally resolving aliases could be harmful. For
> example, if www.example.net and ftp.example.net are both aliases for
> mail.example.net, rewriting them could lose useful information.
>
> 8.8. Header Rewriting
>
> The MSA MAY rewrite local parts and/or domains, in the envelope and
> optionally in address fields of the header, according to local
> policy. For example, a site may prefer to rewrite 'JRU' as '
> J.Random.User' in order to hide logon names, and/or to rewrite '
> squeeky.sales.example.net' as 'zyx.example.net' to hide machine names
> and make it easier to move users.
>
> However, only addresses, local-parts, or domains which match specific
> local MSA configuration settings should be altered. It would be very
> dangerous for the MSA to apply data-independent rewriting rules, such
> as always deleting the first element of a domain name. So, for
> example, a rule which strips the left-most element of the domain if
> the complete domain matches '*.foo.example.net' would be acceptable.

draft-gellens-submit-bis-00.txt has the same text, also in 8.7 and 8.8.

[Charles Lindsey]

In <***@erols.com> Bruce Lilly <***@erols.com> writes:

>Not just the (Bcc recipient) user; also to remind the author (via his
>file copy) to whom he sent bccs (perhaps some time after the fact).

For those MUAs which maintain a 'sent' folder (not all do), that would be
a sensible place to keep the full Bcc set.

--
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: [phone] Fax: [phone] Web: http://www.cs.man.ac.uk/~chl
Email: ***@clerew.man.ac.uk Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9 Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

[Venkat Ramisetty]

Hi,

I would like to know whether we can map a message composed with OLE
attachments to MIME complaint?

Regards,
Venkat

[Keith Moore]

> But the bottom line is that software uses bcc for two very different
> functions: to tell another email transport-related entity to send a
> bcc, or to tell the user that he has received a bcc.

While I agree, I don't see how the dual use of Bcc is worse than the
dual uses of the To and Cc fields.

[Charles Lindsey]

In <***@brandenburg.com> Dave Crocker <***@dcrocker.net> writes:

>Folks,

>I think Nathaniel's note does an excellent job of representing a popular
>line of thinking about these issues. It's critical of that line of
>thinking, rather than of Nathaniel...

>We need to make sure that our assertions about human communications
>issues have some relationship to the normal, non-computer range of human
>behaviors. Unfortunately in the world of email standards,tend to be both
>arbitrary and restrictive in those assertions.


>NB> The essence of a "bcc" is "I want a copy to go to this person, but I
>NB> don't want the 'real' recipient to know it." Given that

>Nit pick: The usual term is 'primary', rather than 'real'.


>NB> motivation/intent, writing it down in a header field is a somewhat
>NB> dubious concept to begin with.

>The concept of making such a note for the author's agent, such as a
>secretary, is not dubious at all. It is the only way the author can
>tell the agent how to handle the message.


>NB> Anyone who puts in a "bcc" field
>NB> pretty obviously intends for it to be taken out before it reaches the
>NB> ultimate recipient.

>In fact that did not used to be standard practise.

>The benefit of having a BCC field in a message is that the recipient
>does not need to guess why they got the message. (Lest anyone says 'if i
>got the message but am not listed in it then i am obviously a bcc
>recipient' we need to observe both the baroqueness (baroquity?) of such
>a (non-)labeling technique and its potential to misbehave.)

>In fact it is not unreasonable to have each bcc recipient get a copy of
>the message with a bcc header that contains (only) their name in it.

In that case, it would be reasonable for an MTA to remove a Bcc header
that did not agree with the envelope address. Whether that would be
expressed as a MAY, SHOULD or MUST is another matter of course :-( .

>In some scenarios, it would even be reasonable to have all the bcc
>recipients get the same copy and have it list all of them. This is a
>way to keep a special (closed) group informed about some public
>activity, and permit them to continue private discussions about it.

Nice, but not so easy to implement.

>NB> This suggests, however, an obvious way to handle
>NB> such headers: Take them out of the headers and (probably) inject them
>NB> into the SMTP recipient stream as soon as you possibly can!

Indeed so. With the understanding that any MUA which uses the '-t' option
of its MTA is simply subcontracting part of its MUA activity, and it had
better be aware of exactly what the MTA has been subcontracted to do
(actually, use of the '-t' option of an MTA is more likely to occur in
special-purpose mailing scripts than in regular general-purpose MUAs).

--
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: [phone] Fax: [phone] Web: http://www.cs.man.ac.uk/~chl
Email: ***@clerew.man.ac.uk Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9 Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

[Keith Moore]

>> In fact it is not unreasonable to have each bcc recipient get a copy
>> of
>> the message with a bcc header that contains (only) their name in it.
>
> In that case, it would be reasonable for an MTA to remove a Bcc header
> that did not agree with the envelope address. Whether that would be
> expressed as a MAY, SHOULD or MUST is another matter of course :-( .

Actually it's occasionally quite useful to be able to let two Bcc:
recipients know that they've both been sent the email. It's just that
the typical MUA is quite inflexible in letting the user determine
exactly which Bcc: recipients get to see which Bcc addresses.

Keith

[Tony Hansen]

Charles Lindsey wrote:

> Dave Crocker <***@dcrocker.net> writes:
>>In fact it is not unreasonable to have each bcc recipient get a copy of
>>the message with a bcc header that contains (only) their name in it.
>
> In that case, it would be reasonable for an MTA to remove a Bcc header
> that did not agree with the envelope address. Whether that would be
> expressed as a MAY, SHOULD or MUST is another matter of course :-( .

No, this is not reasonable at all. Consider the case of a bcc recipient
whose email is forwarded. The envelope address won't agree with the bcc
header, but you still don't want the bcc header touched.

It's a simple rule: the MTA and MSA should only touch the headers their
respective protocols say they should be adding or touching. And that
list is very small.

Tony Hansen
***@att.com

[Nathaniel Borenstein]

Everyone seems to have simple rules, and yet I will freely confess that
I do not have any coherent model at this point for how bcc is supposed
to work. The more I hear the more confused I get.

I think that someone who thinks they have a coherent mental model for
all of this should try to write it down. -- Nathaniel

On Aug 16, 2004, at 10:15 AM, Tony Hansen wrote:

>
> Charles Lindsey wrote:
>
> > Dave Crocker <***@dcrocker.net> writes:
>>> In fact it is not unreasonable to have each bcc recipient get a copy
>>> of
>>> the message with a bcc header that contains (only) their name in it.
>> In that case, it would be reasonable for an MTA to remove a Bcc header
>> that did not agree with the envelope address. Whether that would be
>> expressed as a MAY, SHOULD or MUST is another matter of course :-( .
>
> No, this is not reasonable at all. Consider the case of a bcc
> recipient whose email is forwarded. The envelope address won't agree
> with the bcc header, but you still don't want the bcc header touched.
>
> It's a simple rule: the MTA and MSA should only touch the headers
> their respective protocols say they should be adding or touching. And
> that list is very small.
>
> Tony Hansen
> ***@att.com
>
>
>

[Arnt Gulbrandsen]

Nathaniel Borenstein writes:
> I think that someone who thinks they have a coherent mental model for
> all of this should try to write it down.

Rule no. 1: If you don't want a given recipient to see a particular
header field, don't send it in the first place.

an arrogant
Arnt

[Kai Henningsen]

***@guppylake.com (Nathaniel Borenstein) wrote on 16.08.04 in <1E6D888B-EF99-11D8-9641-***@guppylake.com>:

> Everyone seems to have simple rules, and yet I will freely confess that
> I do not have any coherent model at this point for how bcc is supposed
> to work. The more I hear the more confused I get.

On the other hand, I see an overwhelming majority arguing for the exact
same model - occasionally offering that some particular extension might
make sense, but not actually arguing it should be done.

> I think that someone who thinks they have a coherent mental model for
> all of this should try to write it down. -- Nathaniel

It's *only* the MUA's job to make sure everyone gets the right Bcc; header
or absence of same. One particular way to implement it is via
/usr/{sbin,lib}/sendmail -t, or any other kind of contract with a
different piece of software that is then considered part of the MUA
implementation. The safest way is not handing that responsibility off, but
sending separate copies whenever differences in Bcc: are wanted.

Nobody else ought to touch it.

MfG Kai

[ned+]

> > In that case, it would be reasonable for an MTA to remove a Bcc header
> > that did not agree with the envelope address. Whether that would be
> > expressed as a MAY, SHOULD or MUST is another matter of course :-( .

> No, this is not reasonable at all. Consider the case of a bcc recipient
> whose email is forwarded. The envelope address won't agree with the bcc
> header, but you still don't want the bcc header touched.

In the case of an MTA this is quite correct - forwarding makes it completely
impractical. X.400 made the mistake of thinking envelope-header correlation of
this sort was feasble as part of its MDN design. The result was that MDNs were
a crapshoot in X.400 - sometimes they worked, other times they didn't, and
often as not even exports had difficulty figuring out why. We really don't want
to go here.

Another issue with this is that it creates another condition under which the
MTA would have to perform splitting. (The reasons it does so should be
obvious.) Depending on overall MTA architecture additional splitting cases can
add real complexity, which is bad.

A more interesting case is that of bcc removal by a MSA. My guess is that the
number of cases where an MUA applies different rewrites to header and envelope
so they don't match is fairly small. So having an MSA perform bcc removal is a
considerably safer proposition.

However, this ignores another use-case: That of a file-copy listing all bcc's
that were sent. And the splitting issue remains. There's also the very real
possibility of creep out of the SUBMISSION space.

It's a much closer call, but I don't think this is worth it even if it were
restricted to SUBMISSION and MSAs.

> It's a simple rule: the MTA and MSA should only touch the headers their
> respective protocols say they should be adding or touching. And that
> list is very small.

Yep. Simple and powerful and perilous to mess with.

Ned

[Jacob Palme]

At 10.15 -0400 04-08-16, Tony Hansen wrote:
>It's a simple rule: the MTA and MSA should only touch the headers their respective protocols say they should be adding or touching. And that list is very small.

If someone wants the MTA to remove Bcc headers when
forwarding a message, and to split a message depending on
whether the recipients are in BCC or not, they should get
IETF to standardize a new ESMTP extension to control this
feature from the MUA in communication with the MTA.
--
Jacob Palme <***@dsv.su.se> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/jpalme/

[Jacob Palme]

At 11.44 -0400 04-08-13, Nathaniel Borenstein wrote:
>As one who has always despised the whole concept of bcc's, everything I say should be taken with a large grain of salt. However...

Bcc actually has two functions:

(1) To hide recipients.

(2) To stop replies, i.e. working as a kind of "Non-Reply-To".

Most MUAs interpret "Reply-To" to mean "Send replies
intended for the originator of this message here", and thus
the common command "Reply-to-all" usually sends a reply to
all addresses in "To", "Cc" and "Bcc". But since non-Bcc
recipients will not see any "Bcc", their
reply-to-all-replies are not sent to "Bcc" recipients.

For example, I sometimes send a copy of a message I have
sent to my boss, but want to protect my boss from seeing
all future messages in a possibly long thread of future
discussion on an issue. To achieve this, I use "Bcc".

I thus do not actually want to hide the fact that my boss
is getting a copy, but I have to use "Bcc" to protect my
boss from getting a long thread of messages.

If IETF had standardized a new header "Group-Reply-To"
(similar to "Follow-Up-To" in Usenet News), and if all
mailers would by default use these names when people used
the "reply-to-all" command, this would have been better.
But, as many of us remember, IETF never could reach
consensus on "Group-Reply-To", and thus "Bcc" is needed to
achieve the effect described above.

Note: Some of you may say that it is the task for the
recipients, not for me, to decide whether my boss should
get a copy of their replies. However,most mail users just
use the "reply-to-all" command in their mailers, not
thinking of who the individual recipients of their reply
will be.
--
Jacob Palme <***@dsv.su.se> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/jpalme/

[Keith Moore]

> Bcc actually has two functions:
>
> (1) To hide recipients.
>
> (2) To stop replies, i.e. working as a kind of "Non-Reply-To".

good point. I've certainly used Bcc: in that way. of course, it has
the unintended side-effect of hiding those recipients from view by
other recipients.

> If IETF had standardized a new header "Group-Reply-To"
> (similar to "Follow-Up-To" in Usenet News), and if all
> mailers would by default use these names when people used
> the "reply-to-all" command, this would have been better.

IMHO, To-noreply: and CC-noreply: fields would be far better. sending
MUAs would use them to generate envelope recipients just as if they
were To and Cc. receiving MUAs would display them, but not send
replies to them.

Keith

[Dave Crocker]

JJP> Bcc actually has two functions:
...
JP> (2) To stop replies, i.e. working as a kind of "Non-Reply-To".

While I think it would be entirely reasonable for an rMUA to take the
presence of BCC as meaning that the rMUA should not allow a reply
command to automatically include TO or CC, I do not believe that it a
common rMUA behavior. In fact, I've never heard of it.

In the earliest version of MH (pre-Rose), the posting command sent the
BCC copy separately from the To/CC copy. One of the things is did was
to make To and CC be [To] and [CC]. So they looked like normall to and
cc fields, but the reply commands did not see them present.

This got removed from later versions of MH and I haven't see anything
similar since then.

Accidentally copying to the to and cc folks is certainly an important
issue.


d/
--
Dave Crocker <dcrocker-at-brandenburg-dot-com>
Brandenburg InternetWorking <www.brandenburg.com>
Sunnyvale, CA USA <tel:[phone]>

[Tony Hansen]

Dave Crocker wrote:

> JJP> Bcc actually has two functions:
> ...
> JP> (2) To stop replies, i.e. working as a kind of "Non-Reply-To".
>
> While I think it would be entirely reasonable for an rMUA to take the
> presence of BCC as meaning that the rMUA should not allow a reply
> command to automatically include TO or CC, I do not believe that it a
> common rMUA behavior. In fact, I've never heard of it.
>
> In the earliest version of MH (pre-Rose), the posting command sent the
> BCC copy separately from the To/CC copy. One of the things is did was
> to make To and CC be [To] and [CC]. So they looked like normall to and
> cc fields, but the reply commands did not see them present.
>
> This got removed from later versions of MH and I haven't see anything
> similar since then.
>
> Accidentally copying to the to and cc folks is certainly an important
> issue.

Dave, I think the behaviour in early-MH that you describe is different
from Jacob's suggested use of Bcc.

He's suggesting a form of To/Cc that other people cannot accidentally
reply to. These people will receive a copy of your message, but will not
see replies. He doesn't mind that the To/Cc people know that these other
people have received a copy, so Bcc isn't really quite what he wants.
Right now, the only way to achieve this type of behaviour is to use Bcc
and then put some text in the message saying "I've also bcc'd this
message to x, y and z".

If I understand correctly, your description of early-MH's behaviour is
that the copy of the message that the BCC recipients receive would have
their To/Cc headers zapped such that the BCC recipient will see who got
the message, but cannot do a reply.

Tony Hansen
***@att.com

[Dave Crocker]

Tony,


TH> He's suggesting a form of To/Cc that other people cannot accidentally
TH> reply to.

That's what I meant to describe. The headers sent by the early MH for a
bcc were:

Date: xxx
[To]: yyy
[cc]: zzz
Subject: aaa
From: bbb

So a regular reply command would not see the primary or secondary
address lists.


TH> These people will receive a copy of your message, but will not
TH> see replies. He doesn't mind that the To/Cc people know that these other
TH> people have received a copy, so Bcc isn't really quite what he wants.

I had missed that. So he wants these "tertiary" recipients to be listed
explicitly, but to get copies that cannot be automatically replied to.
In other words, they are outside the fishbowl of the conversation. They
get to listen but not participate.

Certainly that is a reasonable mode of 'participation'. At the least, it would take
a new address field for the special class of recipients.

We probably don't need to disable the to/cc fields, for them, because
the semantics of the new, third field would be "do not allow a Reply
command to to/cc". Of course, any MUA could choose to violate that
standard...



d/
--
Dave Crocker <dcrocker-at-brandenburg-dot-com>
Brandenburg InternetWorking <www.brandenburg.com>
Sunnyvale, CA USA <tel:[phone]>

[Bruce Lilly]

Dave Crocker wrote:

> That's what I meant to describe. The headers sent by the early MH for a
> bcc were:
>
> Date: xxx
> [To]: yyy
> [cc]: zzz
> Subject: aaa
> From: bbb

Gack! I'm assuming "early" means before RFC 2822. Was it before RFC 822?
RFC 822 required at least one destination address header field (RFC 2822
has relaxed that, and it was not required in RFCs 733 and 724), and there
are none in that example!

[Dave Crocker]

Bruce,

BL> Dave Crocker wrote:

>> That's what I meant to describe. The headers sent by the early MH for a
>> bcc were:
>>
>> Date: xxx
>> [To]: yyy
>> [cc]: zzz
>> Subject: aaa
>> From: bbb

BL> Gack! I'm assuming "early" means before RFC 2822. Was it before RFC 822?

Yes. It was right around the time of RFC733, plus or minus a few months.


BL> RFC 822 required at least one destination address header field (RFC 2822
BL> has relaxed that, and it was not required in RFCs 733 and 724), and there
BL> are none in that example!

yeah. it probably violated that (later?) requirement, but it would be
easy to hack in a pro-forma To.

d/
--
Dave Crocker <dcrocker-at-brandenburg-dot-com>
Brandenburg InternetWorking <www.brandenburg.com>
Sunnyvale, CA USA <tel:[phone]>

[Kai Henningsen]

***@dcrocker.net (Dave Crocker) wrote on 16.08.04 in <***@brandenburg.com>:


TH>> He's suggesting a form of To/Cc that other people cannot accidentally
TH>> reply to.

> That's what I meant to describe. The headers sent by the early MH for a
> bcc were:
>
> Date: xxx
> [To]: yyy
> [cc]: zzz
> Subject: aaa
> From: bbb
>
> So a regular reply command would not see the primary or secondary
> address lists.

Ah, no. The suggestion was NOT for the Bcc: recipients to be unable to
reply, it was for the *non*-Bcc: recipients being unable to reply *to* the
Bcc: recipients (by simply not having those addresses in the headers they
see).

You could call that a one-off or snapshot copy.

> I had missed that. So he wants these "tertiary" recipients to be listed
> explicitly, but to get copies that cannot be automatically replied to.

No, no, no, no. See above. You still got it the wrong way around.

> In other words, they are outside the fishbowl of the conversation. They
> get to listen but not participate.

The whole point is they do *not* get to listen.

MfG Kai

[Dave Crocker]

Tony,


TH> He's suggesting a form of To/Cc that other people cannot accidentally
TH> reply to.

That's what I meant to describe. The headers sent by the early MH for a
bcc were:

Date: xxx
[To]: yyy
[cc]: zzz
Subject: aaa
From: bbb

So a regular reply command would not see the primary or secondary
address lists.


TH> These people will receive a copy of your message, but will not
TH> see replies. He doesn't mind that the To/Cc people know that these other
TH> people have received a copy, so Bcc isn't really quite what he wants.

I had missed that. So he wants these "tertiary" recipients to be listed
explicitly, but to get copies that cannot be automatically replied to.
In other words, they are outside the fishbowl of the conversation. They
get to listen but not participate.

Certainly that is a reasonable mode of 'participation'. At the least, it would take
a new address field for the special class of recipients.

We probably don't need to disable the to/cc fields, for them, because
the semantics of the new, third field would be "do not allow a Reply
command to to/cc". Of course, any MUA could choose to violate that
standard...



d/
--
Dave Crocker <dcrocker-at-brandenburg-dot-com>
Brandenburg InternetWorking <www.brandenburg.com>
Sunnyvale, CA USA <tel:[phone]>

[Tony Hansen]

Dave Crocker wrote:

> TH> These people will receive a copy of your message, but will not
> TH> see replies. He doesn't mind that the To/Cc people know that these other
> TH> people have received a copy, so Bcc isn't really quite what he wants.
>
> I had missed that. So he wants these "tertiary" recipients to be listed
> explicitly, but to get copies that cannot be automatically replied to.
> In other words, they are outside the fishbowl of the conversation. They
> get to listen but not participate.

My understanding is that these tertiary recipients would only see the
initial message of the conversation. It's a visible form of Bcc.

Tony Hansen
***@att.com

[Dave Crocker]

Tony,

TH> My understanding is that these tertiary recipients would only see the
TH> initial message of the conversation. It's a visible form of Bcc.

To nit pick: Then it's not a bcc in any fashion. Current bcc's can reply
(though they are not supposed to) and current bcc's are not seen by the
to/cc recipients.

Still, there is certainly nothing wrong with having a visible set of
addressees who are to be copied but who also are being told to keep
their digital mouths shut.

That said, I think this wanders down the path of the jillions of
interesting ideas one might implement, all of which sound interesting
and only some of which are useful to the general Internet user
population.

What's the market pull for this one?

d/
--
Dave Crocker <dcrocker-at-brandenburg-dot-com>
Brandenburg InternetWorking <www.brandenburg.com>
Sunnyvale, CA USA <tel:[phone]>

[Kai Henningsen]

***@dcrocker.net (Dave Crocker) wrote on 21.08.04 in <***@brandenburg.com>:

TH>> My understanding is that these tertiary recipients would only see the
TH>> initial message of the conversation. It's a visible form of Bcc.

> To nit pick: Then it's not a bcc in any fashion. Current bcc's can reply
> (though they are not supposed to) and current bcc's are not seen by the
> to/cc recipients.
>
> Still, there is certainly nothing wrong with having a visible set of
> addressees who are to be copied but who also are being told to keep
> their digital mouths shut.

Really, I fail to understand what is so hard to understand here, but it
seems you consistently refuse to get the point.

Let's try again.

The scenario is this:

From: A
Subject: Current state of project X
To: B, C
Bcc: Boss

with the (intended) result that replies-to-all from B and C will not reach
Boss.

*Nobody* is "told to keep their digital mouths shut". B and C are told
"don't bother Boss, let's keep any discussion to A, B, and C".

Boss, of course, is free to decide to start a discussion of his own; it
just isn't expected that he'll want to - the expectation is that he'll
wait for the next status report.

MfG Kai

[Arnt Gulbrandsen]

Hi,

there's an RFC covering smarthosts now, 2476. It hasn't been very widely
deployed, but IMO no good MUA can be blatantly incompatible with it.

RFC 2476 doesn't suggest that the message submission servers hould remove bcc.

For this reason, I think that a MUA which wants bcc removed cannot trust
the smarthost to do so. It must do the job itself.

Philip Hazel writes:
> 4. Among the MUAs, I think that Pine never sends Bcc: lines, but Mutt by
> default does, assuming that the MTA will deal with them (Mutt does
> not use the -t option).

IMNSHO, mutt is in the wrong.

Whether exim is right is a different question. I guess that boils down
to whether you prefer to Do Things Right or add workarounds for
brokenware.

Arnt

[ned+]

> there's an RFC covering smarthosts now, 2476. It hasn't been very widely
> deployed, but IMO no good MUA can be blatantly incompatible with it.

> RFC 2476 doesn't suggest that the message submission servers hould remove bcc.

> For this reason, I think that a MUA which wants bcc removed cannot trust
> the smarthost to do so. It must do the job itself.

I agree with this assessment. An MUA that cares about bcc handling has no
choice but to do it itself, if for no other reason than it cannot count on it
being done elsewhere. I happen to think the transport infrastructure shouldn't
muck with bcc, but whether it is correct or not really doesn't change the
reality for an MUA.

> Philip Hazel writes:
> > 4. Among the MUAs, I think that Pine never sends Bcc: lines, but Mutt by
> > default does, assuming that the MTA will deal with them (Mutt does
> > not use the -t option).

> IMNSHO, mutt is in the wrong.

I agree.

Although I recognize that customers can and will demand that MSAs/MTAs/MTAs
be able to remove bcc and have provided the means to do it in my own
code, I regard it as a misconfiguration to actually set things up this way.

Ned

[Tony Hansen]

Philip Hazel wrote:

> The issue of who handles Bcc: header lines has again been raised, and I
> am seeking opinions as widely as I can. The requirement is
> straightforward: non-Bcc recipients of a message should not see the
> addresses of any Bcc recipients, that is, their copies of the message
> should not contain Bcc: header lines.

The MUA, however it is constituted, should handle all BCC processing,
and the cloning of messages with and without the Bcc: header.

IMO, each Bcc: recipient should receive a copy with the Bcc: header for
themself and only themself. No other copies should mention that Bcc:
recipient.

Tony Hansen
***@att.com

[Jacob Palme]

Another related issue is whether "Message-ID" should be the
same or different, when a message is sent to some
recipients with "Bcc" headers and to other recipients
without.

Arguments for the same Message-ID: This means that threads
will be intact for all recipients whose mailers use the
Message-ID to keep track of threads.

Arguments agains the same Message-ID: Many mailers suppress
multiple copies of the same message with the same
Message-ID. Thus, if a person receives two copies of a
message, one with and one without Bcc, then depending on
the order of reciept, only one of them will be retained.

Of course, the ideal mailer, which suppresses multiple
copies of a message, should merge the headers of the
multiple copies, but this is not very commonly done.
--
Jacob Palme <***@dsv.su.se> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/jpalme/

[Bruce Lilly]

Jacob Palme wrote:

> Arguments agains the same Message-ID: Many mailers suppress
> multiple copies of the same message with the same
> Message-ID.

That's not necessarily always an argument against keeping the
same msg-id -- some recipients *like* having effective
duplicate suppression.

[Claus Färber]

Philip Hazel <***@cus.cam.ac.uk> schrieb/wrote:
> 1. Exim (which I maintain) removes Bcc: lines if, and only if, it is
> called with the -t option. In that case, it is constructing an
> envelope from the header data, and IMHO in doing so it is fulfilling
> an MUA function. In all other cases it leaves Bcc: lines alone.

The /usr/[s]bin/sendmail interface is a proprietary interface introduced
by Sendmail. So all other applications providing this interface should
be as close to Sendmail's behaviour as possible, i.e. they should remove
Bcc headers.

Philosophically, the mailers are still doing the MUA's job by sanitizing
the message and (finally) handling it over to the message router (MTA).
It just happens that that part of Unix MUAs is usually shipped along
with the MTA.

Claus
--
http://www.faerber.muc.de

[Kai Henningsen]

***@faerber.muc.de (Claus Färber) wrote on 17.08.04 in <***@gmane.3247.org>:

> Philip Hazel <***@cus.cam.ac.uk> schrieb/wrote:
> > 1. Exim (which I maintain) removes Bcc: lines if, and only if, it is
> > called with the -t option. In that case, it is constructing an
> > envelope from the header data, and IMHO in doing so it is fulfilling
> > an MUA function. In all other cases it leaves Bcc: lines alone.
>
> The /usr/[s]bin/sendmail interface is a proprietary interface introduced
> by Sendmail. So all other applications providing this interface should
> be as close to Sendmail's behaviour as possible, i.e. they should remove
> Bcc headers.

... when using the -t option.

> Philosophically, the mailers are still doing the MUA's job by sanitizing
> the message and (finally) handling it over to the message router (MTA).
> It just happens that that part of Unix MUAs is usually shipped along
> with the MTA.

And in fact, it would be entirely possible to implement a
/usr/sbin/sendmail that implemented *only* that part of the functionality,
handing messages off to a separate MTA via SMTP or SUBMISSION.

MfG Kai

[Jacob Palme]

At 20.56 +0200 04-08-17, Kai Henningsen wrote:
>... when using the -t option.

I do not understand this "-t" option. "-t" looks to me like
a Unix shell command parameter. I thought that MUAs
communicated with MTAs for outgoing mail using the SMTP
protocol or a variant of SMTP adapted for communication
between an MUA or the first MTA, and not by using a
unix shell interface.

Is "-t" an ESTMP facility? How, otherwise, can the MUA tell
the MTA that it wants the MTA to remove Bcc headers from
non-Bcc recipients?
--
Jacob Palme <***@dsv.su.se> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/jpalme/

[Bruce Lilly]

Jacob Palme wrote:
> At 20.56 +0200 04-08-17, Kai Henningsen wrote:
>
>>... when using the -t option.
>
>
> I do not understand this "-t" option. "-t" looks to me like
> a Unix shell command parameter.

It is an argument supplied to the program when executed.

> I thought that MUAs
> communicated with MTAs for outgoing mail using the SMTP
> protocol or a variant of SMTP adapted for communication
> between an MUA or the first MTA, and not by using a
> unix shell interface.

That's one way. Running an external program (usually on the
same host) is another method sometimes used, particularly by
older MUAs.

> Is "-t" an ESTMP facility?

No.

> How, otherwise, can the MUA tell
> the MTA that it wants the MTA to remove Bcc headers from
> non-Bcc recipients?

By invoking the external program with the specified argument
and providing the message as input.

Of course, not everybody wants to run (much less administer
and maintain) sendmail on every host with an MUA. Nor is it
practical when run on a different host via remote execution
(user names.ids might not match, there are security implications,
etc.).

[Philip Hazel]

On Wed, 18 Aug 2004, Jacob Palme wrote:

> I do not understand this "-t" option. "-t" looks to me like
> a Unix shell command parameter. I thought that MUAs
> communicated with MTAs for outgoing mail using the SMTP
> protocol or a variant of SMTP adapted for communication
> between an MUA or the first MTA, and not by using a
> unix shell interface.

Not true. Some use SMTP, others call the MTA directly by running the
command /usr/[s]bin/sendmail in a shell interface. (That was, I believe,
the original method, before we had MUAs that ran on PCs.)

> Is "-t" an ESTMP facility? How, otherwise, can the MUA tell
> the MTA that it wants the MTA to remove Bcc headers from
> non-Bcc recipients?

Using SMTP (or the SUBMIT variant), it can't. That's the whole point.

Regards,
Philip

--
Philip Hazel University of Cambridge Computing Service,
***@cus.cam.ac.uk Cambridge, England. Phone: [phone].

[ned+]

> At 20.56 +0200 04-08-17, Kai Henningsen wrote:
> >... when using the -t option.

> I do not understand this "-t" option. "-t" looks to me like
> a Unix shell command parameter. I thought that MUAs
> communicated with MTAs for outgoing mail using the SMTP
> protocol or a variant of SMTP adapted for communication
> between an MUA or the first MTA, and not by using a
> unix shell interface.

Some do, many do not. The lack of a standarized submission mechanism for most
of the time RFC822/SMTP has existed has led to a large number of ad-hoc schemes
being developed. Some are true MSA mechanisms while others actually implement a
split MUA. (And some are configurable and can do both.)

Some schemes are proprietary and only operate between one specific pair of
entities, others are more open. Submission can be done over protocol (not
necessarily TCP - I've seen DECnet, X.25, and even X.29 used), it can be done
via a subroutine library, or it can be done through a pipe. I'm sure there are
many other variants as well, and I haven't even touched on non-RFC822-based
email here.

Ned

[Nick Ing-Simmons]

Jacob Palme <***@dsv.su.se> writes:
>At 20.56 +0200 04-08-17, Kai Henningsen wrote:
>>... when using the -t option.
>
>I do not understand this "-t" option. "-t" looks to me like
>a Unix shell command parameter.

It is.

>I thought that MUAs
>communicated with MTAs for outgoing mail using the SMTP
>protocol or a variant of SMTP adapted for communication
>between an MUA or the first MTA, and not by using a
>unix shell interface.

They do.

But in Unix world MUA-Part1 can commumicate with MUA-Part2
via a pipe, like:

cat message | /usr/lib/sendmail -t

This is using cat/sendmail pair as a MUA.

sendmail instance could do MTA job as well,
or could open socket to ANOTHER instance of sendmail as MTA
and talk SMTP over that socket.

>
>Is "-t" an ESTMP facility?

No.

>How, otherwise, can the MUA tell
>the MTA that it wants the MTA to remove Bcc headers from
>non-Bcc recipients?

The summary just posted explained that MTA has no bussiness
removing Bcc headers. It is a MUA function that happens, on
Unix, to be implemented by another instance of sendmail
binary invoked a particular way.

[Jacob Palme]

At 16.32 +0100 04-08-18, Nick Ing-Simmons wrote:
>The summary just posted explained that MTA has no bussiness
>removing Bcc headers. It is a MUA function that happens, on
>Unix, to be implemented by another instance of sendmail
>binary invoked a particular way.

This is obviously the consensus of the discussion.

But it is not unreasonable that a consenting MUA and MTA
may agree to move part of the burden of the MUA to the MTA,
like for examples is already accepted that the initital MTA
may add a "Date" header for a message lacking such a
header. Since MTA meddling with Bcc is highly
controversial, this should of course never be done unless
there is explicit agreement between MUA and MTA to do it
this way. "-t" might be a way of indicating such an
agreement, but only works when submission is done by
execution, which is probably not the most common method of
MUA-MTA communication.

If there is to be a method for an MUA to ask an MTA to
meddle with Bcc, then it is important for this
communication between MUA and MTA is standardized, so that
misunderstandings will not occur, since misunderstandings
can result in embarassing situations.

Do not interpret the above to mean that I feel such a
standard to be very urgent or important, nor that I am
willing to be editor of it.

I have not been very active in standards work recent
years, but I remember there has been discussion of
writing a special standard on what the first receiving
MTA, when getting a message from an MUA, is allowed
to do with the message, such as adding "Date" and
"Message-ID". Was such a standard ever developed?
it should be the place for a discussion of a possible
"Bcc"-meddling ESMTP extension.
--
Jacob Palme <***@dsv.su.se> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/jpalme/

[Arnt Gulbrandsen]

Jacob Palme writes:
> I have not been very active in standards work recent years, but I
> remember there has been discussion of writing a special standard on
> what the first receiving MTA, when getting a message from an MUA, is
> allowed to do with the message, such as adding "Date" and
> "Message-ID". Was such a standard ever developed?

I guess 2476 section 8 may be what you remember.

Arnt

[ned+]

> At 16.32 +0100 04-08-18, Nick Ing-Simmons wrote:
> >The summary just posted explained that MTA has no bussiness
> >removing Bcc headers. It is a MUA function that happens, on
> >Unix, to be implemented by another instance of sendmail
> >binary invoked a particular way.

> This is obviously the consensus of the discussion.

> But it is not unreasonable that a consenting MUA and MTA
> may agree to move part of the burden of the MUA to the MTA,

It is not unreasonable for two entities to agree to such a thing. But these
entities are no longer what we call an MUA and MTA.

> like for examples is already accepted that the initital MTA
> may add a "Date" header for a message lacking such a
> header.

Actually, it isn't accepted for an MTA to do this. It can be done by an MSA,
which is a different beast.

> Since MTA meddling with Bcc is highly
> controversial, this should of course never be done unless
> there is explicit agreement between MUA and MTA to do it
> this way. "-t" might be a way of indicating such an
> agreement, but only works when submission is done by
> execution, which is probably not the most common method of
> MUA-MTA communication.

-t causes sendmail to in effect assume part of the role of an MUA. It most
moves things way past what an MTA can or should be doing.

> If there is to be a method for an MUA to ask an MTA to
> meddle with Bcc, then it is important for this
> communication between MUA and MTA is standardized, so that
> misunderstandings will not occur, since misunderstandings
> can result in embarassing situations.

It would be possible to define a means to do this as part of our MSA SUBMISSION
protocol, but as you say it would have to be negotiated and therefore would in
no way make the present behavior of agents that blithely assume the transport
infrastructure will take care of of bcc issues. These agents are simply broken
and need to be fixed.

And since such an extension would be just that: An extension, MUAs for
the forseeable future could not assume the extension's presence. This means
they'll have to have their own bcc code.

This in turn begs the question of whether or not there is any real advantage to
having such an extension. It turns out there is: An MUA that is either memory
or bandwidth limited could push the message splitting activities over to the
MSA server and avoid having to send the same message multiple times. The price
we pay for this is having code to handle bcc in two places - a considerable
increase in complexity.

> Do not interpret the above to mean that I feel such a
> standard to be very urgent or important, nor that I am
> willing to be editor of it.

I for one have a great deal of difficulty accessing the real need for various
extensions that attempt to cater to the needs of memory or bandwidth limited
clients. Sure, some of these are nobrainers: If you want a cell phone to be
able to forward a feature-length film you gonna need a forward-without-download
facility along the lines of what the LEMONADE WG is developing, and I don't see
that need changing in the forseeable future. But the need for many other
possible extensions, including this one, are not nearly as obvious. I'm just
not sure whre the sweet spot really is.

Ned

[Philip Hazel]

On Mon, 17 Aug 2004, Claus Färber wrote:

> The /usr/[s]bin/sendmail interface is a proprietary interface introduced
> by Sendmail. So all other applications providing this interface should
> be as close to Sendmail's behaviour as possible, i.e. they should remove
> Bcc headers.

In the case of -t, yes. I disagree in other cases because in those
situations it is not known to be a submission. The message may have come
in via UUCP, or be coming back from a virus scanner or mailing list
exploder.

Said G.K. Chesterton in "The Defendant": "'My country, right or wrong,'
is a thing that no patriot would think of saying. It is like saying, 'My
mother, drunk or sober'".

I think the same sentiment applies to Sendmail as to mothers.

Besides, people tell me that different versions/configurations of
Sendmail behave differently, though I have no direct evidence to support
this, so it may or may not be true.

Regards,
Philip

--
Philip Hazel University of Cambridge Computing Service,
***@cus.cam.ac.uk Cambridge, England. Phone: [phone].

[Philip Hazel]

Summary of the Bcc discussion: posted to exim-users, exim-dev, & ietf-822

There were a few responses on the exim-users mailing list. There was a
longer discussion on the ietf-822 mailing list, to which a number of
people who have worked in email protocol development for many years
contributed. These included Keith Moore (author of several RFCs), Dave
Crocker (author of RFC 822), and Tony Hansen (another RFC author).

Some more data points were posted: Smail-3 behaves like Exim; Elm and
Mulberry behave like Pine (they do their own Bcc: processing).


1. The substantive point

With one possible exception, everybody's opinion was that an MTA has no
business messing with Bcc: header lines. Here is a quote from Tony
Hansen:

"The MUA, however it is constituted, should handle all BCC processing,
and the cloning of messages with and without the Bcc: header."

This confirms my belief that Exim's default behaviour is the correct
one. I hope this particular issue is now laid to rest once and for all.
I have saved all the responses just in case it arises again in the
future.

The possible exception was a post containing this paragraph:

The /usr/[s]bin/sendmail interface is a proprietary interface
introduced by Sendmail. So all other applications providing this
interface should be as close to Sendmail's behaviour as possible, i.e.
they should remove Bcc headers.

This is the "my country, right or wrong" argument. There was one
response that added "... when -t is specified", and that is also my
feeling. A call to /usr/sbin/sendmail without -t is not guaranteed to be
a submission. Indeed, the post continued:

Philosophically, the mailers are still doing the MUA's job by
sanitizing the message and (finally) handling it over to the message
router (MTA). It just happens that that part of Unix MUAs is usually
shipped along with the MTA.

and I would argue that without -t, you cannot, and indeed should not,
assume that the sending entity is an MUA that is submitting a new
message (see further discussion below).


2. Other discussion

Some interesting points of clarification came up in the discussion. The
functions of mail handling are now seen as split into four distinct
parts, as follows:

MUA -> MSA -> MTA -> MDA -> MUA

(user agent, submission agent, transfer agent, delivery agent). There is
nothing to say that each of these functional distinctions has to be
implemented by a different program. Thus, it is generally agreed that a
program handling a local incoming message using the "-t" command line
option is acting partly as an MUA (constructing the envelope) and partly
as an MSA (accepting a new message submission). If it goes on to deliver
the message over SMTP it is performing an MTA function. If it delivers
into a local mailbox it is performing an MDA function.

There was some opinion expressed that an MSA might be somewhere that
Bcc: header lines could be processed. As this is not sanctioned by RFC
2476 (for MUA->MSA delivery over a network), it would be necessary to
define an extension to the SUBMIT protocol so that the MUA could request
the MSA to do that processing. In the case of a submission via the
command line, the -t option is already available for this purpose.

When -t is not used, a general-purpose MTA such as Exim cannot guess
that it is being called as an MSA. It has to be configured as such, for
example by testing conditions such as "submitted from a host on the
local LAN" or "submitted via port 587". If an extension to SUBMIT were
defined, another test could be "requested by the SUBMIT protocol". (It
is quite easy to implement such tests in Exim and (for example) remove
Bcc: lines based on the result, but this has to be an explicit
configuration setting; it cannot be defaulted.)

Note that "submitted via the command line" (in the absence of -t) is not
good enough as a test for an MSA. Such messages may have come from
another email regime (e.g. UUCP - some people do still use it) or from a
virus scanner or a mailing list exploder.


3. Uses of Bcc

As part of the discussion, a list of uses of bcc was enumerated. There
are at least four major ones:

(a) As an instruction from the end user to the MUA to send a copy of the
message to one or more addresses, without telling the To: and Cc:
recipients.

(b) When present in a copies sent to bcc recipients, to inform them why
they received the message (and, if more than one bcc recipient is
listed, to inform them about additional bcc addressees). This should
alert bcc recipients to take care when replying.

(c) To prevent unwanted group replies where the sender wants to send a
copy of a message to someone (whether or not the other recipients
should know about this), but not subject that person to lots of
replies from the other recipients who blindly "reply-to-all".

(d) When saved in the sender's sent-mail folder, a Bcc: header acts as a
record of who the message was sent to.

Is that overloaded or is that overloaded? :-)

Regards,
Philip

--
Philip Hazel University of Cambridge Computing Service,
***@cus.cam.ac.uk Cambridge, England. Phone: [phone].

[Bill McQuillan]

On Wed, 2004-08-18 at 03:36:24, Philip Hazel <***@cus.cam.ac.uk> wrote:

<snip>

> With one possible exception, everybody's opinion was that an MTA has no
> business messing with Bcc: header lines. Here is a quote from Tony
> Hansen:

> "The MUA, however it is constituted, should handle all BCC processing,
> and the cloning of messages with and without the Bcc: header."

<snip>

I haven't seen it mentioned here yet, but shouldn't the rule be that the
agent that extracts the contents of the Bcc: header field, along with To:
and Cc:, in order to construct the 2821 RCPT TO: commands, is the agent that
determines the contents and/or (lack of) presence of the Bcc: field in the
message. Any agent that receives the recipient addresses via RCPT TO: can
be assured that the decision has already been made.

--
Bill McQuillan <***@pobox.com>

[ned+]

> On Wed, 2004-08-18 at 03:36:24, Philip Hazel <***@cus.cam.ac.uk> wrote:

> <snip>

> > With one possible exception, everybody's opinion was that an MTA has no
> > business messing with Bcc: header lines. Here is a quote from Tony
> > Hansen:

> > "The MUA, however it is constituted, should handle all BCC processing,
> > and the cloning of messages with and without the Bcc: header."

> <snip>

> I haven't seen it mentioned here yet, but shouldn't the rule be that the
> agent that extracts the contents of the Bcc: header field, along with To:
> and Cc:, in order to construct the 2821 RCPT TO: commands, is the agent that
> determines the contents and/or (lack of) presence of the Bcc: field in the
> message. Any agent that receives the recipient addresses via RCPT TO: can
> be assured that the decision has already been made.

The part here about receiving addresses via RCPT TO is certainly fine. I
believe the part about agents which construct envelopes from headers is also OK
- it is certainly how I've implemented it. However, since in our model this
sort of processing happens within the MUA, it isn't clear to be we can, or for
that matter should, specify it.

Ned

[Tony Hansen]

Excellent summary, Philip!

Just for completeness, here is an excerpt from the sendmail man page,
describing the -t option.

-t Read message for recipients. To:, Cc:, and Bcc: lines
will be scanned for people to send to. The Bcc: line
will be deleted before transmission. Any addresses in
the argument list will be suppressed. The NoRecipien-
tAction Processing Option can be used to change the
behaviour when no legal recipients are included in the
message.

Note that this strictly affects how the sendmail program is used from
the command line and reading from standard input, and does not affect
how it works when reading input using the SMTP protocol. (There is NO
sendmail option which tells it to look at To/Cc/Bcc when it is using the
SMTP protocol.)

Tony Hansen
***@att.com

Philip Hazel wrote:

>
> Summary of the Bcc discussion: posted to exim-users, exim-dev, & ietf-822
>
> There were a few responses on the exim-users mailing list. There was a
> longer discussion on the ietf-822 mailing list, to which a number of
> people who have worked in email protocol development for many years
> contributed. These included Keith Moore (author of several RFCs), Dave
> Crocker (author of RFC 822), and Tony Hansen (another RFC author).
>
> Some more data points were posted: Smail-3 behaves like Exim; Elm and
> Mulberry behave like Pine (they do their own Bcc: processing).
>
>
> 1. The substantive point
>
> With one possible exception, everybody's opinion was that an MTA has no
> business messing with Bcc: header lines. Here is a quote from Tony
> Hansen:
>
> "The MUA, however it is constituted, should handle all BCC processing,
> and the cloning of messages with and without the Bcc: header."
>
> This confirms my belief that Exim's default behaviour is the correct
> one. I hope this particular issue is now laid to rest once and for all.
> I have saved all the responses just in case it arises again in the future.
>
> The possible exception was a post containing this paragraph:
>
> The /usr/[s]bin/sendmail interface is a proprietary interface
> introduced by Sendmail. So all other applications providing this
> interface should be as close to Sendmail's behaviour as possible, i.e.
> they should remove Bcc headers.
>
> This is the "my country, right or wrong" argument. There was one
> response that added "... when -t is specified", and that is also my
> feeling. A call to /usr/sbin/sendmail without -t is not guaranteed to be
> a submission. Indeed, the post continued:
>
> Philosophically, the mailers are still doing the MUA's job by
> sanitizing the message and (finally) handling it over to the message
> router (MTA). It just happens that that part of Unix MUAs is usually
> shipped along with the MTA.
>
> and I would argue that without -t, you cannot, and indeed should not,
> assume that the sending entity is an MUA that is submitting a new
> message (see further discussion below).
>
>
> 2. Other discussion
>
> Some interesting points of clarification came up in the discussion. The
> functions of mail handling are now seen as split into four distinct
> parts, as follows:
>
> MUA -> MSA -> MTA -> MDA -> MUA
>
> (user agent, submission agent, transfer agent, delivery agent). There is
> nothing to say that each of these functional distinctions has to be
> implemented by a different program. Thus, it is generally agreed that a
> program handling a local incoming message using the "-t" command line
> option is acting partly as an MUA (constructing the envelope) and partly
> as an MSA (accepting a new message submission). If it goes on to deliver
> the message over SMTP it is performing an MTA function. If it delivers
> into a local mailbox it is performing an MDA function.
>
> There was some opinion expressed that an MSA might be somewhere that
> Bcc: header lines could be processed. As this is not sanctioned by RFC
> 2476 (for MUA->MSA delivery over a network), it would be necessary to
> define an extension to the SUBMIT protocol so that the MUA could request
> the MSA to do that processing. In the case of a submission via the
> command line, the -t option is already available for this purpose.
>
> When -t is not used, a general-purpose MTA such as Exim cannot guess
> that it is being called as an MSA. It has to be configured as such, for
> example by testing conditions such as "submitted from a host on the
> local LAN" or "submitted via port 587". If an extension to SUBMIT were
> defined, another test could be "requested by the SUBMIT protocol". (It
> is quite easy to implement such tests in Exim and (for example) remove
> Bcc: lines based on the result, but this has to be an explicit
> configuration setting; it cannot be defaulted.)
>
> Note that "submitted via the command line" (in the absence of -t) is not
> good enough as a test for an MSA. Such messages may have come from
> another email regime (e.g. UUCP - some people do still use it) or from a
> virus scanner or a mailing list exploder.
>
>
> 3. Uses of Bcc
>
> As part of the discussion, a list of uses of bcc was enumerated. There
> are at least four major ones:
>
> (a) As an instruction from the end user to the MUA to send a copy of the
> message to one or more addresses, without telling the To: and Cc:
> recipients.
>
> (b) When present in a copies sent to bcc recipients, to inform them why
> they received the message (and, if more than one bcc recipient is
> listed, to inform them about additional bcc addressees). This should
> alert bcc recipients to take care when replying.
>
> (c) To prevent unwanted group replies where the sender wants to send a
> copy of a message to someone (whether or not the other recipients
> should know about this), but not subject that person to lots of
> replies from the other recipients who blindly "reply-to-all".
>
> (d) When saved in the sender's sent-mail folder, a Bcc: header acts as a
> record of who the message was sent to.
>
> Is that overloaded or is that overloaded? :-)
>
> Regards,
> Philip
>

[Bruce Lilly]

Philip Hazel wrote:
>
> Summary of the Bcc discussion: posted to exim-users, exim-dev, & ietf-822
>
> There were a few responses on the exim-users mailing list. There was a
> longer discussion on the ietf-822 mailing list,
[...]
> 2. Other discussion
>
> Some interesting points of clarification came up in the discussion. The
> functions of mail handling are now seen as split into four distinct
> parts, as follows:
>
> MUA -> MSA -> MTA -> MDA -> MUA
>
> (user agent, submission agent, transfer agent, delivery agent).

Question: Is there an RFC where these terms as used here are defined?
I couldn't find one (in particular, the only reference to "MDA" that
I was able to find was in RFC 3528, which has nothing to do with this
topic).

Aside: I am aware of diagrams in RFCs 2821 and 3464, but they only
show part of the chain; 2821 deals only with SMTP client/server with
no distinction among MSA/MTA/MDA, and 3464 deals only with "MTA"s. I
am also aware of Patrik Faltstrom's "mailflows" diagram, but again
that simply refers to everything as "MTA".

Additions: Typically there is a message store of some sort into which
the MDA places messages. There may also be a message access server
(MAS) between that message store and the recipient MUA. The MAS is
typically a POP variant or IMAP server. And of course gateways to/
from non-SMTP/POP/IMAP mail exist.

[Cyrus Daboo]

Hi Bruce,

--On Friday, September 3, 2004 7:57 AM -0400 Bruce Lilly <***@erols.com>
wrote:

> Question: Is there an RFC where these terms as used here are defined?
> I couldn't find one (in particular, the only reference to "MDA" that
> I was able to find was in RFC 3528, which has nothing to do with this
> topic).

<http://www.ietf.org/internet-drafts/draft-crocker-email-arch-01.txt>

That is a work in progress - obviously.

--
Cyrus Daboo

[Dave Crocker]

Folks,


CD> <http://www.ietf.org/internet-drafts/draft-crocker-email-arch-01.txt>
CD> That is a work in progress - obviously.

Thanks for mentioning this, Cyrus.

In fact I would like to progress the document, but have not gotten
enough feedback to justify claiming that it represents community
consensus.

PLEASE comment on it. The first round of comments resulted in
substantial changes. I'm more than happy to have that happen again
(as long as it looks like convergence, of course.)

Then again, a wave of "looks fine"s won't be met with hostility,
either...


d/
--
Dave Crocker <dcrocker-at-brandenburg-dot-com>
Brandenburg InternetWorking <www.brandenburg.com>
Sunnyvale, CA USA <tel:[phone]>