





                                FetchMail v1.21 

                              by Richard Murray









The obligatory notice:   This  software has got diddly/squat to do with  Argo. 
                         If it dies,  don't ring the support line. Instead sit 
                         down and write me a nice votriolic flame. Then, write 
                         me another email describing what happened.

PS:                      You  can say  anything  in your  flame.  I've  survived 
                         boarding school, I can take it. ;^)

PPS:                     THIS SOFTWARE IS NO LONGER SUPPORTED.
                         However, if you have something to say, contact me at:
                            heyrick -at- merseymail -dot- com
                            [correct Summer 2007]


 



IMPORTANT:           If you have been using a previous version of the FetchMail 
                    preprocessor,  please read this user guide carefully.  The 
                    preprocessor has been rewritten, differently...!



 INTRODUCTION: 

     This software is really two seperate components in one program.  We shall 
     hereby  treat the software in this way,  as there is little  relationship 
     between the two parts.




 PART ONE - FETCHMAIL: 

     Everybody  wants  a  way to be able to save out their  email  file  in  a 
     readable format.
     Argo,  understandably,  are  none  too thrilled at the  idea  of  telling 
     everybody how their mail scrambling system works...

     So  people  start  hacking the mail  system.  It's  a  brilliant  idea... 
     Twiddle  this  and twiddle that.  But it can cause great  confusion  when 
     things  go wrong.  Imagine also if the next mail system was written in  C 
     and all your email files were unencrypted?

     What  is needed is some kind of addition to perform the mail decoding  as 
     required.  Something that is as secure as DTposty,  if not more  so,  and 
     doesn't let just any old person loose on the email files....

     ...something like  FetchMail .


     FetchMail reads which users are defined (up to a maximum of  ten),  which 
     users  have  which mailboxes,  and password protection  (if  enabled)  is 
     applied throughout.



     To use FetchMail:

          1. Load it up and click on the iconbar icon that appears.

          2. Select a mailbox using the up/down arrows.

          3. Enter the password, if appropriate.

          4. Select which mailboxes you would like retrieved. The types are as 
             follows:
               "Standard" - The  mailbox you  see  when  you  enter  the  mail 
                            system.
               "Saved"    - Where mail goes when you click the filing  cabinet 
                            icon.
               "Log"      - A log of your outgoing messages.

          5. Select which of the strip options you require.  Unfortunately the 
             "Strip cute babes in a Beverley Hills teledrame"  option has  not 
             been coded yet. :-)


 
             "Strip  Binaries"  will  remove  UUcoded   attachments  and  MIME 
             attachments.

             "Strip  headers" will remove the routing  garbage at the  top  of 
             each  message.  A side effect  is that you won't  know  who  each 
             message came from!  A later version of FetchMail will  optionally 
             add a short-form header.

             "Strip signatures" will strip everything from the "-- " marker to 
             the beginning of the next message.  This may well include  binary 
             attachments.

          6. Enter the pathname in the icon or drag the directory icon the the 
             destination.  The  base name is  always your  email  prefix  (for 
             example "rmurray" or "adopt" or "xina" et cetera).

             Upon dragging,  the save widget will automagically be kicked into 
             action.  If you don't drag-save,  then click the  "Fetch my mail" 
             icon.

             "Cancel" quits.

     All mail is stored in a directory based upon your username.  If you would 
     like something different,  use the writable icon.  Inside this  directory 
     are  up  to  three  files  named  "Standard",   "Saved"  or  "Log"  which 
     corresponds to your selected options.

     The  decoding  process may take some time,  so an hourglass  will  appear 
     giving  you  a percentage complete count.  I'd estimate  approximately  a 
     megabyte  in  fifteen seconds on an ARM 3 using a  medium-slow  harddisc. 
     Because of the way FetchMail works, it has to decode a line at a time.


 PART TWO - THE PREPROCESSOR: 

     This  is a powerful addition to the basic FetchMail which was based on  a 
     burning  desire to filter out the crap that somehow manages to  find  its 
     way to you.  Chain letters,  adverts for 1-800 downloads of naked  girls, 
     timeshare and some truly amazing dross.  Also there was a want to provide 
     an automatic acknowledgement facility.

     FetchMail can be set up to preprocess incoming mail automatically by  the 
     use  of  something called the "FetchMail prodder".  This runs  as  a  VIX 
     extension and looks for mail download complete broadcasts. To install the 
     VIX support, simply copy the directory within the "VIX" directory of your 
     FetchMail  archive into the "!Voyager.VIX" directory.  There is no  fancy 
     installation  procedure due to the use of subdirectories,  which the  VIX 
     installer isn't too keen on.

      INSTALLATION IS FAIRLY INVOLVED. 


     After  using FetchMail 1.1x for several months,  it became  more  trouble 
     than it was worth to check you were not auto-acking SPAM.  I also  wanted 
     NOT  to reject spam forwarded from my "arcticbb.demon.co.uk"  address  as 
     that could be indirectly giving the spammers my new address. Additionally 
     to that I often loaded my email into !Edit to see who sent what.

     The answer to all of this was to recreate the processor as an interactive 
     system.  The  wetware (your brain) is the only thing that will  know  for 
     sure that a spam is a spam.


     Firstly, let's set up the VIX to auto-run FetchMail when your email fetch 
     completes:

          1.   Inside the !FetchMail application is a directory "VIX".  Inside 
               the "VIX" directory is a file called "!InstallVX".

          2a.  If Voyager is or has been loaded since the last reset,  double-
               click on the "!InstallVX" file.

          2b.  If Voyager has not been run yet,  copy the "FetchVIX" directory 
               into "!Voyager.VIX".

     The  FetchMail Prodder VIX has been installed.  It will not  take  effect 
     until next time Voyager is loaded.

     The purpose of FetchMail Prodder is to wait for email fetching to finish. 
     When this is done, it will automatically run FetchMail. FetchMail Prodder 
     shuts itself down when you disconnect from the Internet.







 
 Using the preprocessor: 

     When  the preprocessor is invoked,  you will briefly see a window with  a 
     sliding bar.  During this stage,  FetchMail is assessing the new  emails. 
     The preprocessor obey file automatically backs up your email file  within 
     the same directory as "Email", with the name "_In_Email".


     When that is finished, a window will open:




                    You have 8 emails waiting to be read.
                  (4 replies; 0 acks; 0 adverts; 2 binaries)

               Would you like to look through your email?     

          Yes                                               No



     Clicking "No" will restore email files and quit the preprocessor with  no 
     effects.

     Clicking "Yes" will open a window showing the first valid  email.  Emails 
     configured  to be automatically killed,  rejected or bounced will not  be 
     shown  to you.  Additionally,  emails that you have already seen will  be 
     skipped over.

     The  skip process does not poll.  If you have let your mail back up  then 
     this might take some time.  If your computer freezes,  don't reset it  if 
     your harddisc is still being accessed.

     If  this  window does not open,  no valid emails were found  (either  the 
     whole lot has been dealt with or you have already read them all).



     New message received

     A new message has been received from an unknown sender.
     What do you want to do with this message?


           To : [Angela Chase                     ]     [ ] Reply  [ ] Ack
         From : [Richard Murray                   ]        [ View header ]
      Subject : [Where have you been, all my so-called life!             ]
       Begins : [Dear Angela,                                            ]
                [                                                        ]

     [ Okay it]    [ Keep it ]    [ Bounce it ]    [ Reject it]    [ Kill it ]



 
     This  shows you who the message is from and who it is to.  You  are  also 
     shown the subject and the first two lines of the message; which should be 
     enough to give you the gist of what the message is going to be about.

     For  example,  if it is from somebody with an unpronouncable address  and 
     the  subject  "Hello"  then it might be  somebody  writing  to  you.  If, 
     however, the first line is the old favourite "Our research shows that you 
     are  interested  in our promotion",  then you can be pretty  sure  it  is 
     another (unresearched) spam!

     The 'Reply' and 'Ack' buttons indicate a valid acknowledgement or reply.

     Buttons you can click:

          View headers   This  will  open the message headers so you  can  see 
                         who/what sent you the message.

          Okay it        Will accept the message (copy it back to the incoming 
                         email file) and add the email address to the list  of 
                         recognised senders.

          Keep it        Will  accept the message but will NOT add  the  email 
                         address to the list of recognised senders.

          Bounce it      Will  not  accept the message.  Instead  it  will  be 
                         bounced  in  full  (with whatever  message  you  have 
                         written  into  the "bouncehdr" file tacked  onto  the 
                         top).  Useful for people that annoy you - but DO  NOT 
                         use this with spammers.

          Reject it      This is the one to use with spammers.  If emulates  a 
                         mailer daemon saying that you do not exist within the 
                         domain.  A  good  spambot should recognise  this  and 
                         automatically  remove  you from  the  spam  list.  Of 
                         course, the entire message is quoted back.
                         Rejection  implies  that you never  want  to  receive 
                         anything from this address ever again.  So it will be 
                         written  into the user actions file to always  reject 
                         further messages from this user.


     This will repeat for each email in your email file.  When  finished,  the 
     files   will  be  tidied  and  windows  closed  and   preprocessor   quit 
     automatically.



 Configuration files: 

     Configuration files are located within the !FetchMail.Data directory.  We 
     shall examine each file in turn...

     AutoAckHdr
          Not currently used.

 
     BounceHdr
          This header is attached to the beginning of bounced  messages.  Make 
          it as rude as you like. <grin>
          The entire received message is quoted below this text.


     headers
          When scanning messages, the current headers are written to this file 
          so they are available when you click "View headers".


     MessageIDs
          When  scanning,  IDs of scanned messages are written here so you  do 
          not see the same message twice.


     Options
          This  file is only for compatibility.  Later released  of  FetchMail 
          will use a different configuration system.


     signoffs
          These  files  are  used  to sign  off  bounced  and  Ack'd  messages 
          according to the following naming convention:

          Scanning left to right through the email name, stop at "." or "@" or 
          after ten characters. For example:
               rmurray@argonet.co.uk              "rmurray"
               claire.danes@argonet.co.uk         "claire"
               reallylongname@argonet.co.uk       "reallylong"
          [note: the rmurray address is no longer valid, hasn't been for YEARS]


     UserAction
          This is the most important file, it determines what action should be 
          taken for recognised email addresses.

          The first line is a counter of how many entries are in the file.
          The second line is a version number. This should read '1'.

          Remaining lines are laid out as "<action> <address>".
          The following actions are recognised:

               O    Okay  (allow through,  will disable reject and  bounce  on 
                    this message).
               B    Bounce this message. FetchMail does not write this option, 
                    but you can add it manually.
               R    Reject this message.
               K    Kill this message.  FetchMail does not write this  option, 
                    but you can add it manually.
               !    This  is  a special option for recognised  addresses  that 
                    should  NOT  be  Ack'd,   bounced  or  rejected.   It   is 
                    recommended that you do NOT alter the addresses within the 
                    UserActions file that have this flag.


 
          Email addresses do not use wildcards.  Instead they use the  partial 
          matching method,  where "@argonet.co.uk" will block everything  from 
          the Argonet domain. For example:
               rmurray@argonet.co.uk         - Block mails from the author.
               @argonet.co.uk                - Block mails from Argonet.
               .co.uk                        - Block anything ending '.co.uk'.
               @                             - Block everything (!).
          [note: the rmurray address is no longer valid, hasn't been for YEARS]


 

AND IF IT DIES? 

     FetchMail   preprocessor   copies  your  incoming  email  to   the   file 
     "!FetchMail.Angela".  FetchMail will  refuse  to run if this file exists as 
     this  may contain unread email.  This is in addition to backing  up  your 
     email as "_In_Email".

     FetchMail shouldn't crash. It may moan about something and shut down, but 
     it  shouldn't bomb out with an obscure error,  leaving files open  around 
     the place.

     FetchMail  also  shouldn't  cause  memory leaks or  eat  RMA  as  nothing 
     particularly amazing happens within the code,  but shhhhh...  You are not 
     supposed to know that. It's like the Wizard of Oz. ;^)



 
ACKing: 

     FetchMail currently does not provide ACK facilities.  This will be  added 
     in the near future.



 
ALSO INCLUDED: 

     !FormDec
          An alpha version of a Form decoder.  Doesn't handle text areas  well 
          and most certainly isn't bomb-proof,  but it will take one (and only 
          one)  Argo form CGI message and display it 'smartly' on the  screen. 
          You design a custom template with the information where you want  it 
          and it'll put the information there. Instructions included.









 
 DESIGN SPECIFICATION: 

     FetchMail is written in C and compiled with Acorn C release 4,  with  the 
     WIMP facilities provided by the excellent DeskLib library.

     There is one main 'easter egg', which can be discovered by looking in the 
     code. Getting it to work is left as an exercise for the reader. ;^)

     Version 1.21:  Executable is 52148 bytes.
                    Source code is 92475 bytes





 Warning/Error_Messages: 

     This section details possible warning/error messages and what they  mean. 
     An [F] prefix means it is a fatal error.


      [F]   (unable to claim memory to hold user data structure) 
          Could  not claim memory to hold all the user  definitions.  Increase 
          WimpSlot in !Run or !RunPreprc.

      [F] (I cannot find template definition '<something>') 
          Whatever  template  name  is given in place of  <something>  is  the 
          template  that  could  not be found.  Something is  wrong  with  the 
          Templates file, try reinstalling.

      [F] (Menu descriptor block error) 
          You should not get this message.  If you do,  stop playing with  the 
          software!

      [F] (error trying to read string - couldn't find  <etc...>  )
       or  (error trying to read word - couldn't find  <etc...>  ) 
          This indicates a corruption in your Options or Configure  file.  The 
          file  is written in BASIC's PRINT# format so should  be  repairable. 
          Usually it is easier to delete it and reconfigure.

      [F] (cannot load user definition file!?) 
          Serious problems. The mail reader's "Users" file cannot be opened.

      [F] (unrecognised user definition file format)
           The mail reader's file format is newer than the recognised ones,  or 
          it may be corrupted.

      You have not selected which mailbox to save!
           Click "Standard", "Log" or "Save" as applicable and try again!

 
      Unable to open output file for writing... 
          Cannot save your mailbox,  for some reason the output file cannot be 
          opened. Is the path valid? Is there a locked file already there?

      Couldn't open <whatever> input file - it may not exist yet. 
          The  mailboxes  are created as they are  needed.  For  example,  new 
          mailboxes  may not have a Log mailbox.  This message pops up if  the 
          mailbox cannot be opened - usually because it hasn't been created.

      Your '<whatever>' mailbox is empty, skipping... 
          Again, something that may occur with new mailboxes.

      Unable to open options file for writing... 
          Something is preventing FetchMail from writing it's  configurations. 
          Is the file locked or already open?

      Unable to open options file for reading... 
          Either  the  configurations  file does not  exist  or  something  is 
          preventing it from being opened (is it already opened?).

      Unrecognised option file format - please rebuild. 
          The configurations file format is not recognised. It may be too old, 
          too new or corrupted.

      The file "!FetchMail.angela" exists... If FetchMail crashed last time you 
     use  it,  YOUR EMAIL IS WITHIN THIS FILE.  FetchMail will not  preprocess 
     until this file has been removed.
           Says it all...

      Mail file found, but does not contain valid mail header...
           When reading mail file, expected to find "! rmail", but didn't.

      [F]  (can't reopen the email file for rebuilding!  DO NOT RE-EXECUTE THIS 
     PROGRAM UNTIL YOU HAVE RETRIEVED YOUR CURRENT MAIL...)
           Successfully read and copied the "Email" file,  but could not reopen 
          it   for   update.   Something  weird  is  going   on.   Check   the 
          Imcomming.Email file - should be closed and unprotected.
          Your email is stored within !FetchMail as "angela".

      Email parsing error - unable to find valid ID with input "<something>". 
          Was   unable  to  find  a  valid  email  address  given  the   input 
          <something>. If all else fails, it will look for at least an '@'.

      [F] (cannot add new ID (cannot open 'MessageIDs' file)) 
            Something  is stopping the MessageIDs file from being  opened.  This 
          will be demoted to a non-fatal error in a future version.

      [F] (unable to open user actions file) 
          Couldn't open the user actions file...






 
      [F]  (format  of user actions file is  unrecognised;  expected  type  '1' 
     received type ' <number> ')
           The UserActions file format was unrecognised, it should be:
               <number of entries>
               1
               <repeating for each entry - action code, space, email address>

      [F] (unable to claim memory to hold user actions - increase WimpSlot  and 
     try again) 
          Could  not claim enough memory to load the contents  of  UserActions 
          file (each email address uses 64 bytes).  Increase WimpSlot in  !Run 
          or !RunPreprc and try again.

      [F] ('angela' file missing, lost or dead since last time I looked!?) 
          You  should  not  get this error.  It occurs if  the  'angela'  file 
          vanishes between copying into FetchMail and trying to open it...

      [F]  (unable  to  open DTposty's email file for update -  your  email  is 
     stored within FetchMail as 'angela'...)
           Self evident...
 
     [F] (mail file contains invalid header (at message %d, offset %d))
           Expected  to  find "!  rmail" but didn't.  Gives you  which  message 
          caused  the problem and the offset into the 'angela' file (which  is 
          preserved within !FetchMail).

