Cut-off mails

Use this forum to get help.

Cut-off mails [subject should be "truncated emails"

Postby rdigqd » Thu Mar 11, 2010 9:42 pm

Agreed .... a solution/resolution/'fix' would wonderful.

After a lull in the frequency of 'hits' for a few weeks,
the rate of cut-off/truncated e-mails arriving here
seems to be on an up-swing.
(( Biggest problem on top of that is that I don't ))
(( have a mechanism for at least some of my ))
(( SG ids to route them thru an intermediary so ))
(( that I can get at least one copy of the "full" ))
(( e-mail-plus-any-attached-file(s!) !! ))

Over and above the 'inconvenience' factor, the ramifications
of SG-users not being able to rest assured that they are
receiving the entire text of msgs sent to them is a big
integrity concern and holds potential for a non-trivial
P.R. disaster. (Not to put to fine a point on it, if a
'plain vanilla' ISP's SMTP or POP system 'accidentally'
altered or truncated traffic passing thru them, there'd
be front-page headlines and loud, rude noises. ))

j.

going thru it, there'd
be front-page headlinesc and

e e99
rdigqd
 
Posts: 24
Joined: Mon Feb 21, 2005 7:38 pm

Truncated emails ..

Postby gverelst » Wed Sep 14, 2011 9:42 am

This seems the right topic to post my observations as I'm impacted by truncated mails too. However the post in here seems not that recent. I'll give it a try anyway

I've sent the text below (from a gmail account) as a reply on a spamgourmet mail
Can somebody replay my case with the text inbetween and without the quotes.

"
Natuurlijk wil ik ook van mijn kant het gebruik van 'm mailadres toelichten.
gecre?erd voor
Text as of the line above is truncated in the delivery
"


It's the result a the long original text that I have debugged by stripping the most non relevant elements.
gverelst
 
Posts: 10
Joined: Tue Nov 21, 2006 2:36 pm

Cut-off mails

Postby rdigqd » Wed Sep 14, 2011 12:43 pm

Yes indeeeeeeeed it is "still happening!

Good to see this 'thread' revived by another poster since it
save me from doing it (which I intended to do when I found
a free moment in the next days/week).

A thought for Josh et al. to ponder that might help
in chasing down the "where" of this 'glitch':

I'm no uber-geek expert but I do pick up bits-n-pieces
of info hear and there.

**
** My gut feel is that there is some sort of buffer-overflow like
**situation occurring.
**

If memory serves, when one is manually sending an
email (ie: as in The Old Days of Telnet and so on; not using
a 'client' like Eudora, Thunderbrd, etc.) then the end of the
msg-body text portion of email is indicated by sending a
line/string of text that is simply character "." plus a CR.

IFFFFF there is some buffer-length situation in SG parsing
routines where the body msg text of an email passing thru
just happens to get completely fill the buffer immediately
prior to the ".<CR>" at the end of a sentence in it, ... and SG processing to handle the next part of that sentence (and
the remainder of the msg-body) sees only ".<CR>" this
seems to be the condition that triggers the cut-off/truncated
email scenario.

What happens is that ALLLL of the rest of the email msg body
text _and_ any attached files get 'tossed' away.
I've proven this by sending e-m to an address that then automatically sends the e-m onward to both a 'real' e-m addr
and an SG e-m addr (which then sends the output on to that
same 'real' e-m). That way I have a with/without SG processing side-by-side comparson situation.

Resending the 'truncated' email with the text preceding
that truncation point altered by adding/subtracting some
characters can *_sometimes_* (but rarely) provide a proof-
in-principle of this theory.
I say 'sometimes' cuz all the other "stuff" (headers, etc.)
that composes the actual text-stream of the email item
passing thru the 'net also varies (eg: even if a time-stamps
goes from a single-digit in one of the hhmmss components
t a double-digit value (12:9:33 => 12:11:34) ... thereby
'confusing the test with multiple concurrent 'deltas' in the
length of the text stream that SG actualy processes.

Hope this provides some help/hint (and provides some
impetus/encouragement) towards getting this vexatious
'glitch' squashed in the otherwise FIIIINNNNNNEEEEE
and wonderful SG product.

jim
___________________
rdigqd
 
Posts: 24
Joined: Mon Feb 21, 2005 7:38 pm

Thank You

Postby Karl-Egon » Wed Sep 14, 2011 1:40 pm

Hi Jim,


I am very grateful, for your summary and stimulation of the thread.

I hope imploringly that Josh takes care over again of the problem!

Up to now there was only one solution: Drift to Trashmail...
Nevertheless, this is sad!


Sunny greetings,

Karl-Egon
Last edited by Karl-Egon on Wed Sep 14, 2011 6:48 pm, edited 1 time in total.
das immernoch etwas andere forum:_________ http://berliner-nachrichten.eu.steingrueber.info/phpBB3/
Karl-Egon
 
Posts: 34
Joined: Thu Oct 12, 2006 11:06 am
Location: SOL-3-EU-DE

Mimimal charachters to reproduce the mail truncation ..

Postby gverelst » Wed Sep 14, 2011 6:12 pm

Hi all, I like the interaction ... thanks for your posts
I've drilled down my 'debugging' the mimimum requirement to systematically reproduce the incident at every position in the text
<CRLF>.<CRLF>

Try to sent this (only the text within the quotes)
"
1
.2
3.
.
4
5
"

Or simply (only the text within the quotes)
"

.
text
"

A google search on (only the text within the first and last quotes) might give a more general view on the problematic
"
"<CRLF>.<CRLF>" truncate
"
Let's hope this allows a SG Guru to continue the analysis
Last edited by gverelst on Wed Sep 14, 2011 6:58 pm, edited 1 time in total.
gverelst
 
Posts: 10
Joined: Tue Nov 21, 2006 2:36 pm

Re: Mimimal charachters to reproduce the mail truncation ..

Postby Karl-Egon » Wed Sep 14, 2011 6:48 pm

:D

gverelst wrote: [?]
Try to sent this (only the text within the quotes)
"
1
.2
3.
.
4
5
"

I did. Result (only the text within the quotes):
"
1
.2
3.
"
The rest is truncated.

Or simply (only the text within the quotes)
"

.
text
"
[?]

Result is an empty message. Completly truncated.

:shock:

Who has more results?


Karl-Egon
das immernoch etwas andere forum:_________ http://berliner-nachrichten.eu.steingrueber.info/phpBB3/
Karl-Egon
 
Posts: 34
Joined: Thu Oct 12, 2006 11:06 am
Location: SOL-3-EU-DE

Mimimal charachters to reproduce the mail truncation ...

Postby gverelst » Wed Sep 14, 2011 7:49 pm

Hi Karl-Egon and others who will add their test results :
Pls refer also Which mailclient you used?

My test results :
- Gmail(webmail) -> truncated
- Hotmail(webmail) -> truncated

But:
Lotus notes -> NO trunc ! (my company address)

Does notes use another protocol and thus another stream inside SG? Hopefully this might help to drill down to the root cause

> completed my own post :
Indeed lotus notes uses BASE64 encoding Content-Type: text/plain;
charset=us-ascii
Content-Transfer-Encoding: base64

DQoxDQouMg .....

Hotmail, Gmail use quote-printable Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

1
.2
3.
4
gverelst
 
Posts: 10
Joined: Tue Nov 21, 2006 2:36 pm

Does SG use sendmail, is SG running with the right option?

Postby gverelst » Wed Sep 14, 2011 9:22 pm

Hi all,

I believe SG uses 'sendmail' internally
and I've found this ref:
http://docstore.mik.ua/orelly/networking/tcpip/appe_02.htm

Inside this doc the option -i is described as follow
-i
Normally, an SMTP message terminates when a line containing only a dot is encountered. This argument tells sendmail to ignore the dots in incoming messages.

Or depending on the version it might be another syntax:
http://docstore.mik.ua/orelly/other/Sendmail_3rd/1565928393_ch24-40708.html

O IgnoreDots=bool configuration file (V8.7 and later)
OIgnoreDots=bool command line (V8.7 and later)
define(`confIGNORE_DOTS',bool) mc configuration (V8.7 and later)
-i command-line shorthand (deprecated)
Oibool configuration file (deprecated)
-oibool command line (deprecated)


Isn't that exactly what we are experiencing ?
Would it be that simple ? Can an SG Guru pls check ?
gverelst
 
Posts: 10
Joined: Tue Nov 21, 2006 2:36 pm

Re: Mimimal charachters to reproduce the mail truncation ...

Postby Karl-Egon » Wed Sep 14, 2011 9:49 pm

gverelst wrote:Hi Karl-Egon and others who will add their test results :
Pls refer also Which mailclient you used?

My test results :
- Gmail(webmail) -> truncated
- Hotmail(webmail) -> truncated

But:
Lotus notes -> NO trunc ! (my company address)

My results above:
- Posteo.de (Roundcube webmail) -> truncated

More results:
- Web.de (webmail) -> truncated
- GMX (webmail) -> truncated
- Horde IMP (webmail) -> truncated
- Outlook (local) -> truncated

Normally, an SMTP message terminates when a line containing only a dot is encountered. This argument tells sendmail to ignore the dots in incoming messages.

In the very past time, I noticed, that the message was truncated, if the combination ".<CRLF>" was located at the end of a auto breaked line.
That seems depending from the adjustment of line length on the mails way (not of the sending or receiving mail client).

Maybe it was in reality an auto line break between the last word and the following dot (what never should be separated) of a sentence, followed by a <CRLF>.

The fault is this separation ...
Microsoft wrote:According to Request for Comments (RFC) 821, the character sequence "<CRLF>.<CRLF>", if imbedded inside the body of a message, should be sent transparently through an SMTP server. If the server is not expecting this sequence in the body of a message, the message can be truncated at the point where the "<CRLF>.<CRLF>" combination is entered. Messages sent by CDONTS containing a "<CRLF>.<CRLF>" combination are truncated and not delivered in full. A modification has been made to the Internet Mail Service to ensure these character combinations are passed through correctly.


Sunny greetings,

Karl-Egon
das immernoch etwas andere forum:_________ http://berliner-nachrichten.eu.steingrueber.info/phpBB3/
Karl-Egon
 
Posts: 34
Joined: Thu Oct 12, 2006 11:06 am
Location: SOL-3-EU-DE

Code verification request

Postby gverelst » Thu Sep 15, 2011 10:32 am

Hi
Is the spamgourmet 'production' configuration still in line with the sourceforge code ?
http://spamgourmet.svn.sourceforge.net/ ... amgourmet/
I tried to follow the code. Indeed sendmail is used, but couldn't find the '-i or -OIgnoreDots=true
Most obvious would be to find them in here.

> spamgourmet > conf > spamgourmet.config
48 $mailprogram = '/usr/sbin/sendmail';



> spamgourmet > modules > Mail > Spamgourmet >
CommandLineMailer.pm
package Mail::Spamgourmet::CommandLineMailer;
2 use strict;
3
4 sub new {
5 my $proto = shift;
6 my $class = ref($proto) || $proto;
7 my $self = {};
8 bless $self,$class;
9 my %params = @_;
10 if ($params{'config'}) {
11 $self->{'config'} = $params{'config'};
12 } else {
13 die 'CommandLineMailer must be initialized with an instance of Mail::Spamgourmet::Config.pm';
14 }
15 return $self;
16 }
17
18 sub sendMail {
19 my $self = shift;
20 my $msgref = shift;
21 my $rcpt = shift;
22 my $sender = shift;
23 return if !$rcpt;
24 my $dashf = '';
25 $dashf = " -f\"$sender\"" if $sender;
26 my $mailprogram = $self->{'config'}->getMailProgram();
27 open (MAIL, "|$mailprogram$dashf \"$rcpt\"");
28 print MAIL $$msgref;
29 close MAIL;
30 # $self->{'config'}->debug("sent message to $rcpt");
31 return;
32 }
33
34 1;

Worth to try these? Do you have a testserver running somewhere?
24 my $dashf = ' -i ';
24 my $dashf = ' -OIgnoreDots=true ';
27 open (MAIL, "|$mailprogram$dashf \"$rcpt\"");
gverelst
 
Posts: 10
Joined: Tue Nov 21, 2006 2:36 pm

Info from de developpers forum ....

Postby gverelst » Sun Sep 18, 2011 4:37 pm

Hi all,

See here: seems a similar bug was corrected in the past.
http://bbs.spamgourmet.com/viewtopic.php?p=6600#6600

Can somebody who has access to the code make the effort to verify?
Thanks!
gverelst
 
Posts: 10
Joined: Tue Nov 21, 2006 2:36 pm

Postby josh » Mon Oct 03, 2011 3:24 pm

the call to sendmail is in the config file. We were using ignoredots, but now that's commented out and we're back to not including the option. I'll have to remember why. There may have been a compelling reason.
josh
 
Posts: 1371
Joined: Fri Aug 29, 2003 2:28 pm

Postby chrisarts » Mon Jun 18, 2012 3:49 am

Hey,

I'm digging up this thread because I've ran into the same bug, and it's blocking access to my Apple account. :(

Here's the source of the email:

Code: Select all
[...]
Content-Disposition: inline

Dear -,

To reset your Apple ID password, simply click the link below. That will tak=
e you to a web page where you can create a new password.

Please note that the link will expire three hours after this email was sent=
.

Reset your Apple ID password > <https://iforgot.apple.com/cgi-bin/resetPass=
word.cgi?key=3D90968

[...]


No clue why Apple includes a period on a line in their email :(

Thanks!
chrisarts
 
Posts: 1
Joined: Mon Jun 18, 2012 3:38 am

Previous

Return to Support / Hilfe / ayuda / ondersteuning / ...

Who is online

Users browsing this forum: No registered users and 25 guests