Behaviour I didn't expected
I'm using fetchmail for a few years now and I'm very happy about that. Recently I was doing some experiments with "poll" and "skip" and discovered something I didn't expected.
I have two providers, provider1 and provider2. On provider1 I have a mailbox I check with fetchmail running in deamon mode every 10minutes. This part works fine.
On provider2 I have a mailbox I check manually. I was hoping dat "fetchmail provider2" or "fetchmail -d0 provider2" should be working but it didn't, I only got the mail from provider1.
The only thing that works is "fetchmail -q; fetchmail -d0 provider2; fetchmail". So I first have to kill my deamon, get the mail and start fetching it again.
Is there a better work around?
Btw, I'm using release 6.2.5+NTLM+SDPS+SSL+NLS from debian-sarge.
Fetchmail looses mail
Hello, I think I have found a case where fetchmail will lose mail. If 'keep' is being used to leave mail on the server and the number of messages on the server exceeds the fetchlimit, Fetchmail retrieves up to <fetchlimit> messages, then on the next connection ignores all of the messages, as if they have all been retrieved, when they have not.
I am using uidl, if this makes a difference.
Re: Many Features except Robustness
We've since learned that procmail is also problematic as it loads entire files into memory to process just the headers
curiously i am back here because i just had a big success with getmail where fetchmail failed - a message with a 5Mb attachment (compressed dbase image). procmail handled it fine, although the machine has much more memory, of course.
in a way, it's a privilege to be able to worry about procmail (thanks for the tip) - it was fetchmail's inability to grab an earlier file of this size that finally forced me to use getmail (it's always possible, of course, that's it's a fetchmail config error on my part, but i did check and modify config files with no success). i can also now download channel4 news (uk independent news source) email list, which fetchmail choked on because of incorrect headers.
> I'm using Debian, so installing [getmail] is
> trivial. And it was simple to configure
> - I have it running as a cron job under
> the mail user, piping mail to procmail
> for two users (procmail filters to
> maildirs which are handled by Courier
> imap + Squirrelmail for web access).
> Appears to work fine. You should try
We've since learned that procmail is also problematic as it loads entire files into memory to process just the headers (or that's what I've heard, haven't read the code yet). In these days of large attachments, that can seriously impact server performance, so we've been looking at a few alternatives.
Just so you know, though, the problem is not so serious that we're scheduling downtime to fix it ;) ... we've known about the problem for a long long time, and yet we still run procmail. What surprises me is how the Linux distros (Mandrake at least) still ship with these really bad-example programs as the default configuration.
Re: Many Features except Robustness
I use(d) fetchmail on my own server and had the same frustrating problems (even with relatively low mail levels it would jam every week or so). After reading your comments (thanks!) I realised it wasn't my fault and switched to getmail.
I'm using Debian, so installing is trivial. And it was simple to configure - I have it running as a cron job under the mail user, piping mail to procmail for two users (procmail filters to maildirs which are handled by Courier imap + Squirrelmail for web access). Appears to work fine. You should try it...
Re: HTTP support ?
check out FetchYahoo (http://freshmeat.net/projects/fetchyahoo/)
It's not exactly what you want, but it does the trick.
Many Features except Robustness
First off, we depend on Fetchmail to manage incoming email from several offsite POP/IMAP accounts that must be redistributed locally across three machines; for shear features and ease of deployment, fetchmail is the only game in town.
But it is a major admin headache: Where incoming mail headers are non-standard (which is frequent with spam), or when the MDA barfs on an incoming email (which procmail often does), instead of a graceful recovery, fetchmail unceremoniously aborts.
It has buffer overrun problems too.
When it dies, POP3 leaves all messages in the queue,; it has to complete the run to delete the server messages --- if you've downloaded 188 of 190 and it dies, next time in, you get those same 188 messages, again and again and again until you notice the failure.
In our experience, handling only about 1000 messages a day from 6 sources, fetchmail fails at least two or three times a day: Keep a close watch if you run fetchmail in daemon mode.
Curiously (and fortunately) Eric's earlier program, popclient, is far more resilient to header errors; we use a log-watch (logplay) to monitor syslog for fetchmail failure messages, and launch the ancient popclient to recover. We're seriously contemplating getmail (/projects/getmail/) as an alternative, but it's a beast to install.
Re: HTTP support ?
Knowledge Base Article q259133 (http://support.microsoft.com/support/kb/articles/q259/1/33.ASP) Says:
HTTP mail is based on WebDAV extensions
to the HTTP protocol.
For more information about WebDAV extensions, see
the following Web sites:
Request For Comments (RFC) 2518 (http://www.rfc-editor.org/rfc/rfc2518.txt):
HTTP support ?
Fetchmail rocks, I use it on my Slack 8 box to talk to yahoo.com, exim, InterMail and others, don't get me wrong, I love it.
I must challenge the statement under "About:" on the freshmeat page though:
"It supports every remote-mail protocol now in use on the Internet"
Many HotMail accounts are good for nothing but catching spam. To process this spam to get something done about it you either have to do it manually (ouch! if you're getting 40+ spams a day that's HARD work and lots of dead time!)
To bulk process my spam I have to use Outlook Express to download the mail (it supports a "HTTP Mail Protocol") and then forward them on as attchments to one big spam mail to my Linux box to process them.
This downloading over the modem is very slow, error prone and painful. If fetchmail supported this "HTTP Protocol" for retrieval of mail I wouldn't have to polute my fingers with touching a Windows keyboard, I could cron my spam harvesting, and get to bed earlier at night...
As far as I can see the tricky part to this is the https authentication, because from there on it it appears to just be http. (I may be wrong)
It may be a spawn of the evil empire, but it is a protocol used for remote mail retrieval, and fetchmail doesn't currently support it.
Re: Very good utility ..
It's a very useful utility !
But I think it's too oftently updated (yes, sure !). I'm compiling the sources again and again...
An open, cross-platform journaling program.
A scientific plotting package.