Added new info about Exchange to manual

[offlineimap] / offlineimap.sgml

<!DOCTYPE reference PUBLIC "-//OASIS//DTD DocBook V4.1//EN" [
  <!ENTITY OfflineIMAP "<application>OfflineIMAP</application>">
]>
<!--      "file:///usr/share/sgml/docbook/dtd/xml/4.2/docbookx.dtd"> -->

<reference>
  <title>OfflineIMAP Manual</title>

  <refentry>
    <refentryinfo>
      <address><email>jgoerzen@complete.org</email></address>
      <author><firstname>John</firstname><surname>Goerzen</surname></author>
      <date> $Date: 2004-07-26 10:37:45 -0500 (Mon, 26 Jul 2004) $ </date>
    </refentryinfo>

    <refmeta>
      <refentrytitle>offlineimap</refentrytitle>
      <manvolnum>1</manvolnum>
      <refmiscinfo>John Goerzen</refmiscinfo>
    </refmeta>

    <refnamediv>
      <refname>OfflineIMAP</refname>
      <refpurpose>Powerful IMAP/Maildir synchronization
        and reader support</refpurpose>
    </refnamediv>

    <refsynopsisdiv>
      <cmdsynopsis>
        <command>offlineimap</command>
        <arg>-1</arg>
        <arg>-P <replaceable>profiledir</replaceable></arg>
        <arg>-a <replaceable>accountlist</replaceable></arg>
        <arg>-c <replaceable>configfile</replaceable></arg>
        <arg>-d <replaceable>debugtype[,...]</replaceable></arg>
        <arg>-l <replaceable>filename</replaceable></arg>
        <arg>-o</arg>
        <arg>-u <replaceable>interface</replaceable></arg>
      </cmdsynopsis>
      <cmdsynopsis>
        <command>offlineimap</command>
        <group choice="plain"><arg>-h</arg><arg>--help</arg></group>
      </cmdsynopsis>
    </refsynopsisdiv>

    <refsect1>
      <title>Description</title>

      <para>&OfflineIMAP; is  a  tool  to  simplify  your  e-mail
        reading.  With &OfflineIMAP;, you can read the same mailbox
        from multiple computers.  You get a current copy of your
        messages on each computer, and changes you make one place will be
        visible on all other systems.  For instance, you can delete a message
        on your home computer, and it will appear deleted on your work
        computer as well.  &OfflineIMAP; is also useful if you want to
        use a mail reader that does not have IMAP support, has poor IMAP
        support, or does not provide disconnected operation.
      </para>

      <para>&OfflineIMAP; is <emphasis>FAST</emphasis>; it synchronizes
        my two accounts with over 50 folders in 3 seconds.  Other
        similar tools might take over a minute, and achieve a
        less-reliable result.  Some mail readers can take over 10
        minutes to do the same thing, and some don't even support it
        at all.  Unlike other mail tools, &OfflineIMAP; features a
        multi-threaded synchronization algorithm that can dramatically
        speed up performance in many situations by synchronizing
        several different things simultaneously.
      </para>

      <para>&OfflineIMAP; is <emphasis>FLEXIBLE</emphasis>; you can
        customize which folders are synced via regular expressions,
        lists, or Python expressions; a versatile and comprehensive
        configuration file is used to control behavior; two user
        interfaces are built-in; fine-tuning of synchronization
        performance is possible; internal or external automation is
        supported; SSL and PREAUTH tunnels are both supported; offline
        (or "unplugged") reading is supported; and esoteric IMAP
        features are supported to ensure compatibility with the widest
        variety of IMAP servers.
      </para>

      <para>&OfflineIMAP; is <emphasis>SAFE</emphasis>; it uses an
        algorithm designed to prevent mail loss at all costs.  Because
        of the design of this algorithm, even programming errors
        should not result in loss of mail.  I am so confident in the
        algorithm that I use my own personal and work accounts for
        testing of &OfflineIMAP; pre-release, development, and beta
        releases.  Of course, legally speaking, &OfflineIMAP; comes
        with no warranty, so I am not responsible if this turns out
        to be wrong.
      </para>

      <refsect2>
        <title>Method of Operation</title>
       
        <para>&OfflineIMAP; traditionally 
          operates by maintaining a hierarchy of
          mail folders in Maildir format locally.  Your own mail
          reader will read mail from this tree, and need never know
          that the mail comes from IMAP.  &OfflineIMAP; will detect
          changes to the mail folders on your IMAP server and your own
          computer and bi-directionally synchronize them, copying,
          marking, and deleting messages as necessary.
        </para>
        <para>
          With &OfflineIMAP; 4.0, a powerful new ability has been
          introduced -- the program can now synchronize two IMAP
          servers with each other, with no need to have a Maildir
          layer in-between.  Many people use this if they use a mail
          reader on their local machine that does not support
          Maildirs.  People may install an IMAP server on their local
          machine, and point both &OfflineIMAP; and their mail reader
          of choice at it.  This is often preferable to the mail
          reader's own IMAP support since &OfflineIMAP; supports many
          features (offline reading, for one) that most IMAP-aware
          readers don't.  However, this feature is not as time-tested
          as traditional syncing, so my advice is to stick with normal
          methods of operation for the time being.
        </para>
      </refsect2>
    </refsect1>
    
    <refsect1>
      <title>Quick Start</title>
      <para>If you have already installed &OfflineIMAP; system-wide,
        or your system administrator has done that for you, your task
        for setting up &OfflineIMAP; for the first time is quite
        simple.  You just need to set up your configuration file, make
        your folder directory, and run it!
      </para>

        <para>You can quickly set up your configuration file.  The distribution
          includes a file <filename>offlineimap.conf.minimal</filename>
          (Debian users
        may find this at
          <filename>/usr/share/doc/offlineimap/examples/offlineimap.conf.minimal</filename>) that is a basic example of setting of &OfflineIMAP;.  You can
          simply copy this file into your home directory and name it
          <filename>.offlineimaprc</filename> (note the leading period).  A
          command such as <command>cp offlineimap.conf.minimal ~/.offlineimaprc</command> will do it.  Or, if you prefer, you can just copy this text to
          <filename>~/.offlineimaprc</filename>:
        </para>

        <PROGRAMLISTING>[general]
accounts = Test

[Account Test]
localrepository = Local
remoterepository = Remote

[Repository Local]
type = Maildir
localfolders = ~/Test

[Repository Remote]
type = IMAP
remotehost = examplehost
remoteuser = jgoerzen
</PROGRAMLISTING>

        <para>Now, edit the <filename>~/.offlineimaprc</filename> file with
          your favorite editor.  All you have to do is specify a directory
          for your folders to be in (on the <property>localfolders</property>
          line), the host name of your IMAP server (on the
          <property>remotehost</property> line), and your login name on
          the remote (on the <property>remoteuser</property> line).  That's
          it!</para>

        <para>To run &OfflineIMAP;, you just have to say
          <command>offlineimap</command> -- it will fire up, ask you for
          a login password if necessary, synchronize your folders, and exit.
          See?  You can just throw away the rest of this finely-crafted,
          perfectly-honed manual!  Of course, if you want to see how you can
          make &OfflineIMAP; FIVE TIMES FASTER FOR JUST $19.95 (err, well,
          $0), you have to read on!
        </para>

    </refsect1>

    <refsect1>
      <title>Installation</title>

      <para>If you are reading this document via the "man" command, it is
        likely
        that you have no installation tasks to perform; your system
        administrator has already installed it.  If you need to install it
        yourself, you have three options: a system-wide installation with
        Debian, system-wide installation with other systems, and a single-user
        installation.  You can download the latest version of &OfflineIMAP; from
        <ulink url="http://quux.org/devel/offlineimap/">the &OfflineIMAP;
        website</ulink>.
      </para>

      <refsect2>
        <title>Prerequisites</title>

        <para>In order to use &OfflineIMAP;, you need to have these conditions
          satisfied:
        </para>

        <itemizedlist>
          <listitem>
            <para>Your mail server must support IMAP.  Most Internet Service
              Providers
              and corporate networks do, and most operating systems
              have an IMAP
              implementation readily available.
            </para>
          </listitem>
          <listitem>
            <para>
              You must have Python version 2.2.1 or above installed.
              If you are
              running on Debian GNU/Linux, this requirement will automatically be
              taken care of for you.  If you do not have Python already, check with
              your system administrator or operating system vendor; or, download it from
              <ulink url="http://www.python.org/">the Python website</ulink>.
              If you intend to use the Tk interface, you must have Tkinter
                (python-tk) installed.  If you intend to use the SSL interface, your
                Python must have been built with SSL support.
            </para>
          </listitem>
          <listitem>
            <para>
              Have a mail reader that supports the Maildir mailbox
              format.  Most modern mail readers have this support
              built-in, so you can choose from a wide variety of mail
              servers.  This format is also known as the "qmail"
              format, so any mail reader compatible with it will work
              with &OfflineIMAP;.  If you do not have a mail reader
              that supports Maildir, you can often install a local
              IMAP server and point both &OfflineIMAP; and your mail
              reader at it.
            </para>
        </itemizedlist>
      </refsect2>

      <refsect2>
        <title>System-Wide Installation, Debian</title>
        <para>
          If you are tracking Debian unstable, you may install
          &OfflineIMAP; by simply running the following command as root:
        </para>
        <para>
          <command>apt-get install offlineimap</command>
        </para>
        <para>
          If you are not tracking Debian unstable, download the Debian .deb
          package from the <ulink url="http://quux.org/devel/offlineimap/">&OfflineIMAP; website</ulink>
          and then run <command>dpkg -i</command> to install the downloaded
          package.  Then, skip to <xref linkend="configuration" endterm="configuration-title"> below.  You will type <command>offlineimap</command> to
          invoke the program.
        </para>
      </refsect2>

      <refsect2>
        <title>System-Wide Installation, Other</title>
        <para>
          Download the tar.gz version of the package from the
          <ulink url="http://quux.org/devel/offlineimap/">website</ulink>.
          Then run
          these commands, making sure that you are the "root" user first:
        </para>

        <ProgramListing>tar -zxvf offlineimap_x.y.z.tar.gz
cd offlineimap-x.y.z
python2.2 setup.py install</ProgramListing>
        <para>On some systems, you will need to use
          <command>python</command> instead of <command>python2.2</command>.
          Next, proceed to <xref linkend="configuration" endterm="configuration-title"> below.  You will type <command>offlineimap</command> to
          invoke the program.
        </para>
      </refsect2>

      <refsect2>
        <title>Single-Account Installation</title>
        <para>
          Download the tar.gz version of the package from the
          <ulink url="http://quux.org/devel/offlineimap/">website</ulink>.
          Then run these commands:
        </para>

        <ProgramListing>tar -zxvf offlineimap_x.y.z.tar.gz
cd offlineimap-x.y.z</ProgramListing>

        <para>When you want to run &OfflineIMAP;, you will issue the
          <command>cd</command> command as above and then type
          <command>./offlineimap.py</command>; there is no installation
          step necessary.
        </para>
      </refsect2>
    </refsect1>

    <refsect1 id="configuration">
      <title id="configuration-title">Configuration</title>
      <para>
        &OfflineIMAP; is regulated by a configuration file that is normally 
        stored in <filename>~/.offlineimaprc</filename>.  &OfflineIMAP;
        ships with a file named <filename>offlineimap.conf</filename>
        that you should copy to that location and then edit.  This file is
        vital to proper operation of the system; it sets everything you need
        to run &OfflineIMAP;.  Full documentation for the configuration file
        is included within the sample file.
      </para>
      <para>
        &OfflineIMAP; also ships a file named
        <filename>offlineimap.conf.minimal</filename> that you can also try.
          It's useful if you want to get started with
          the most basic feature set, and you can read about other features
          later with <filename>offlineimap.conf</filename>.
      </para>
    </refsect1>

    <refsect1>
      <title>Options</title>
      <para>
        Most configuration is done via the configuration file.  Nevertheless,
        there are a few command-line options that you may set for
        &OfflineIMAP;.
      </para>

      <variablelist>
        <varlistentry><term>-1</term>
          <listitem><para>Disable most multithreading operations and use
            solely a single-connection
            sync.  This effectively sets the
              <property>maxsyncaccounts</property>
            and all <property>maxconnections</property> configuration file
            variables to 1.
          </para></listitem>
        </varlistentry>
        <varlistentry><term>-P <replaceable>profiledir</replaceable></term>
          <listitem><para>Sets &OfflineIMAP; into profile mode.  The program
            will create <replaceable>profiledir</replaceable>
            (it must not already exist).  As it runs, Python profiling 
            information
            about each thread is logged into profiledir.  Please note: This option
            is present for debugging and optimization only, and should NOT be used
            unless you have a specific reason to do so.  It will significantly
            slow program performance, may reduce reliability, and can generate
            huge amounts of data.  You must use the <option>-1</option> option when
            you use <option>-P</option>.
          </para></listitem>
        </varlistentry>
        <varlistentry><term>-a <replaceable>accountlist</replaceable></term>
          <listitem><para>Overrides the <property>accounts</property> option
            in the <property>general</property> section of the configuration
            file.  You might use this to exclude certain accounts, or to sync
            some accounts that you normally prefer not to.  Separate the
            accounts by commas, and use no embedded spaces.
          </para></listitem>
        </varlistentry>
        <varlistentry><term>-c <replaceable>configfile</replaceable></term>
          <listitem><para>Specifies a configuration file to use in lieu of
            the default, <filename>~/.offlineimaprc</filename>.
          </para></listitem>
        </varlistentry>
        <varlistentry><term>-d <replaceable>debugtype[,...]</replaceable></term>
          <listitem><para>Enables debugging for OfflineIMAP.  This is useful if
            you are trying to track down a malfunction or figure out what is going
            on under the hood.  I suggest that you use this with
            <option>-1</option> to make the results more sensible.</para>

            <para><option>-d</option> requires one or more debugtypes,
              separated by commas.  These define what exactly will be
              debugged, and include three options: <property>imap</property>,
              <property>maildir</property>, and <property>thread</property>.
              The <property>imap</property>
              option will enable IMAP protocol stream and parsing debugging.  Note
              that the output may contain passwords, so take care to remove that
              from the debugging output before sending it to anyone else.  The
              <property>maildir</property> option will enable debugging for
              certain Maildir operations.  And <property>thread</property>
              will debug the threading model.
            </para></listitem>
          </varlistentry>
          <varlistentry><term>-l
          <replaceable>filename</replaceable></term>
            <listitem><para>
              Enables logging to filename.  This will log everything
              that goes to the screen to the specified file.
              Additionally, if any debugging is specified with -d,
              then debug messages will not go to the screen, but
              instead to the logfile only.</para>
          </listitem>
        </varlistentry>
          <varlistentry><term>-o</term>
            <listitem><para>Run only once, ignoring all
              <property>autorefresh</property> settings in the configuration
              file.</para>
            </listitem>
          </varlistentry>
          <varlistentry><term>-h</term> <term>--help</term>
            <listitem><para>Show summary of options.</para></listitem>
          </varlistentry>
          <varlistentry><term>-u <replaceable>interface</replaceable></term>
            <listitem><para>Specifies an alternative user interface module
              to use.  This overrides the default specified in the
              configuration file.  The pre-defined options are listed in
              the User Interfaces section.</para>
            </listitem>
          </varlistentry>
      </variablelist>
    </refsect1>
    <refsect1>
      <title>User Interfaces</title>
      <para>&OfflineIMAP; has a pluggable user interface system that lets you choose how the
        program communicates information to you.  There are two graphical
        interfaces, two terminal interfaces, and two noninteractive interfaces
        suitable for scripting or logging purposes.  The
        <property>ui</property> option in the configuration file specifies
        user interface preferences.  The <option>-u</option> command-line
        option can override the configuration file setting.  The available
        values for the configuration file or command-line are described
        in this section.</para>
      <refsect2>
        <title>Tk.Blinkenlights</title>
        <para>Tk.Blinkenlights is an interface designed to be sleek, fun to watch, and
          informative of the overall picture of what &OfflineIMAP;
          is doing.  I consider it to be the best general-purpose interface in
          &OfflineIMAP;.
        </para>
        <para>
          Tk.Blinkenlights contains, by default, a small window with a row of
          LEDs, a small log, and a row of command buttons.
          The total size of the window is
          very small, so it uses little desktop space, yet it is quite
          functional.  The optional, toggleable, log shows more
          detail about what is happening and is color-coded to match the color
          of the lights.
        </para>
        <para>
          Tk.Blinkenlights is the only user interface that has configurable
          parameters; see the example <filename>offlineimap.conf</filename>
          for more details.
        </para>
        <para>
          Each light in the Blinkenlights interface represents a thread
          of execution -- that is, a particular task that &OfflineIMAP;
          is performing right now.  The colors indicate what task
          the particular thread is performing, and are as follows:
        </para>
        <variablelist>
          <varlistentry>
            <term>Black</term>
            <listitem><para>indicates that this light's thread has terminated; it will light up
              again later when new threads start up.  So, black indicates no
              activity.
            </para></listitem>
          </varlistentry>
          <varlistentry>
            <term>Red (Meaning 1)</term>
            <listitem><para>is the color of the main program's thread, which basically does
              nothing but monitor the others.  It might remind you of HAL 9000 in
              <citation>2001</citation>.
            </para></listitem>
          </varlistentry>
          <varlistentry>
            <term>Gray</term>
            <listitem><para>indicates that the thread is establishing a new connection to the IMAP
              server.
            </para></listitem>
          </varlistentry>
          <varlistentry>
            <term>Purple</term>
            <listitem><para>is the color of an account synchronization thread that is monitoring
              the progress of the folders in that account (not generating any I/O).
            </para></listitem>
          </varlistentry>
          <varlistentry>
            <term>Cyan</term>
            <listitem><para>indicates that the thread is syncing a folder.
            </para></listitem>
          </varlistentry>
          <varlistentry>
            <term>Green</term>
            <listitem><para>means that a folder's message list is being loaded.
            </para></listitem>
          </varlistentry>
          <varlistentry>
            <term>Blue</term>
            <listitem><para>is the color of a message synchronization controller thread.
            </para></listitem>
          </varlistentry>
          <varlistentry>
            <term>Orange</term>
            <listitem><para>indicates that an actual message is being copied.
              (We use fuchsia for fake messages.)
            </para></listitem>
          </varlistentry>
          <varlistentry>
            <term>Red (meaning 2)</term>
            <listitem><para>indicates that a message is being deleted.
            </para></listitem>
          </varlistentry>
          <varlistentry>
            <term>Yellow / bright orange</term>
            <listitem><para>indicates that message flags are being added.
            </para></listitem>
          </varlistentry>
          <varlistentry>
            <term>Pink / bright red</term>
            <listitem><para>indicates that message flags are being removed.
            </para></listitem>
          </varlistentry>
          <varlistentry>
            <term>Red / Black Flashing</term>
            <listitem><para>corresponds to the countdown timer that runs between
              synchronizations.
            </para></listitem>
          </varlistentry>
        </variablelist>
        <para>The name of this interfaces derives from a bit of computer
          history.  Eric Raymond's <citation>Jargon File</citation> defines
          <firstterm>blinkenlights</firstterm>, in part, as:
        </para>
        <blockquote>
          <para>Front-panel diagnostic
            lights on a computer, esp. a dinosaur. Now that dinosaurs are rare,
            this term usually refers to status lights on a modem, network hub, or
            the like.
          </para>
          <para>
            This term derives from the last word of the famous blackletter-Gothic
            sign in mangled pseudo-German that once graced about half the computer
            rooms in the English-speaking world. One version ran in its entirety as
            follows:
          </para>
          <para>
            <emphasis>ACHTUNG!  ALLES LOOKENSPEEPERS!</emphasis>
          </para>
          <para>
            Das computermachine ist nicht fuer gefingerpoken und mittengrabben.
            Ist easy schnappen der springenwerk, blowenfusen und poppencorken
            mit spitzensparken.  Ist nicht fuer gewerken bei das dumpkopfen.
            Das rubbernecken sichtseeren keepen das cotten-pickenen hans in das
            pockets muss; relaxen und watchen das blinkenlichten.
          </para>
        </blockquote>
      </refsect2>

      <refsect2>
        <title>Curses.Blinkenlights</title>
        <para>
          Curses.Blinkenlights is an interface very similar to Tk.Blinkenlights,
          but is designed to be run in a console window (an xterm, Linux virtual
          terminal, etc.)  Since it doesn't have access to graphics, it isn't
          quite as pretty, but it still gets the job done.
        </para>
        <para>Please see the Tk.Blinkenlights section above for more
          information about the colors used in this interface.
        </para>
      </refsect2>

      <refsect2>
        <title>Tk.VerboseUI</title>
        <para>
          Tk.VerboseUI (formerly known as Tk.TkUI) is a graphical interface
          that presents a variable-sized window.  In the window, each
          currently-executing thread has a section where its name and current
          status are displayed.  This interface is best suited to people running
          on slower connections, as you get a lot of detail, but for fast
          connections, the detail may go by too quickly to be useful.  People
          with fast connections may wish to use Tk.Blinkenlights instead.
        </para>
      </refsect2>

      <refsect2>
        <title>TTY.TTYUI</title>
        <para>
          TTY.TTYUI interface is for people running in basic, non-color terminals.  It
          prints out basic status messages and is generally friendly to use on a console
          or xterm.
        </para>
      </refsect2>

      <refsect2>
        <title>Noninteractive.Basic</title>
        <para>
          Noninteractive.Basic is designed for situations in which &OfflineIMAP;
          will be run non-attended and the status of its execution will be
          logged.  You might use it, for instance, to have the system run
          automatically and
          e-mail you the results of the synchronization.  This user interface
          is not capable of reading a password from the keyboard; account
          passwords must be specified using one of the configuration file options.
        </para>
      </refsect2>

      <refsect2>
        <title>Noninteractive.Quiet</title>
        <para>
          Noninteractive.Quiet is designed for non-attended running in situations
          where normal status messages are not desired.  It will output nothing
          except errors and serious warnings.  Like Noninteractive.Basic,
          this user interface
          is not capable of reading a password from the keyboard; account
          passwords must be specified using one of the configuration file options.
        </para>
      </refsect2>

    </refsect1>

    <refsect1>
      <title>Examples</title>
      <para>Here are some example configurations for various situations.
        Please e-mail any other examples you have that may be useful to
        me.
      </para>

      <refsect2>
        <title>Multiple Accounts with Mutt</title>
        <para>
          This example shows you how to set up &OfflineIMAP; to
          synchronize multiple accounts with the mutt mail reader.
        </para>
        <para>
          Start by creating a directory to hold your folders by running
          <command>mkdir ~/Mail</command>.  Then, in your
          <filename>~/.offlineimaprc</filename>, specify:
        </para>
        <programlisting>accounts = Personal, Work</programlisting>
        <para>
          Make sure that you have both an
          <property>[Account Personal]</property>
          and an <property>[Account Work]</property> section.  The
          local repository for each account must have different
          <property>localfolder</> path names.
          Also, make sure
          to enable <property>[mbnames]</property>.
        </para>
        <para>
          In each local repository section, write something like this:
        </para>
        <programlisting>localfolders = ~/Mail/Personal</programlisting>
        <para>
          Finally, add these lines to your <filename>~/.muttrc</filename>:
        </para>
        <programlisting>source ~/path-to-mbnames-muttrc-mailboxes
folder-hook Personal set from="youremail@personal.com"
folder-hook Work set from="youremail@work.com"
set mbox_type=Maildir
set folder=$HOME/Mail
spoolfile=+Personal/INBOX</programlisting>
        <para>
          That's it!
        </para>
      </refsect2>

      <refsect2>
        <title>UW-IMAPD and References</title>
        <para>Some users with a UW-IMAPD server need to use &OfflineIMAP;'s
          "reference" feature to get at their mailboxes, specifying a reference
          of "~/Mail" or "#mh/" depending on the configuration.  The below
          configuration from (originally from docwhat@gerf.org)
          shows using a <property>reference</property> of Mail, a <property>nametrans</property>
          that strips
          the leading Mail/ off incoming folder names, and a
          <property>folderfilter</property> that
          limits the folders synced to just three.
        </para>
        <programlisting>[Account Gerf]
localrepository = GerfLocal
remoterepository = GerfRemote

[Repository GerfLocal]
type = Maildir
localfolders = ~/Mail

[Repository GerfRemote]
type = IMAP
remotehost = gerf.org
ssl = yes
remoteuser = docwhat
reference = Mail
# Trims off the preceeding Mail on all the folder names.
nametrans = lambda foldername: \
            re.sub('^Mail/', '', foldername)
# Yeah, you have to mention the Mail dir, even though it
# would seem intuitive that reference would trim it.
folderfilter = lambda foldername: foldername in [
      'Mail/INBOX',
      'Mail/list/zaurus-general',
      'Mail/list/zaurus-dev',
      ]
maxconnections = 1
holdconnectionopen = no</programlisting>
      </refsect2>

      <refsect2>
        <title>pythonfile Configuration File Option</title>
        <para>You can have &OfflineIMAP;
          load up a Python file before evaluating the
          configuration file options that are Python expressions.  This example
          is based on one supplied by Tommi Virtanen for this feature.
        </para>
        <para>
          In <filename>~/.offlineimap.rc</filename>, he adds these options:
        </para>
        <programlisting>[general]
pythonfile=~/.offlineimap.py
[Repository foo]
foldersort=mycmp</programlisting>
        <para>
          Then, the <filename>~/.offlineimap.py</filename> file will
          contain:
        </para>
        <programlisting>prioritized = ['INBOX', 'personal', 'announce', 'list']

def mycmp(x, y):
   for prefix in prioritized:
       xsw = x.startswith(prefix)
       ysw = y.startswith(prefix)
       if xsw and ysw:
          return cmp(x, y)
       elif xsw:
          return -1
       elif ysw:
          return +1
   return cmp(x, y)

def test_mycmp():
   import os, os.path
   folders=os.listdir(os.path.expanduser('~/data/mail/tv@hq.yok.utu.fi'))
   folders.sort(mycmp)
   print folders</programlisting>
        <para>
          This code snippet illustrates how the <property>foldersort</property>
          option can be customized with a Python function from the
          <property>pythonfile</property> to always synchronize certain
          folders first.
        </para>
      </refsect2>
    </refsect1>
      
    <refsect1>
      <title>Errors</title>
      <para>
        If you get one of some frequently-encountered or confusing errors,
        please check this section.
      </para>
      
      <refsect2>
        <title>UID validity problem for folder</title>
        <para>IMAP servers use a unique ID (UID) to refer to a specific message.
          This number is guaranteed to be unique to a particular message
          <emphasis>forever</emphasis>.
          No other message in the same folder will ever get the same
          UID.  UIDs are an integral part of &OfflineIMAP;'s synchronization
          scheme; they are used to match up messages on your computer to
          messages on the server.
        </para>

        <para>
          Sometimes, the UIDs on the server might get reset.  Usually this will
          happen if you delete and then recreate a folder.  When you create a
          folder, the server will often start the UID back from 1.  But
          &OfflineIMAP; might still have the UIDs from the previous folder by the
          same name stored.  &OfflineIMAP; will detect this condition and skip the
          folder.  This is GOOD, because it prevents data loss.
        </para>

        <para>
          You can fix it by removing your local folder and cache data.  For
          instance, if your folders are under <filename>~/Folders</filename>
          and the folder with the problem is INBOX, you'd type this:
        </para>

        <programlisting>rm -r ~/Folders/INBOX
rm -r ~/.offlineimap/Account-<replaceable>AccountName</>
rm -r ~/.offlineimap/Repository-<replaceable>RepositoryName</></programlisting>

        <para>
          (Of course, replace AccountName and RepositoryName 
          with the names as specified
          in <filename>~/.offlineimaprc</filename>).
        </para>

        <para>Next time you run &OfflineIMAP;, it will re-download
          the folder with the
          new UIDs.  Note that the procedure specified above will lose any local
          changes made to the folder.
        </para>

        <para>
          Some IMAP servers are broken and do not support UIDs properly.  If you
          continue to get this error for all your folders even after performing
          the above procedure, it is likely that your IMAP server falls into
          this category.  &OfflineIMAP; is incompatible with such servers.
          Using &OfflineIMAP; with them will not destroy any mail, but at the same time,
          it will not actually synchronize it either.  (&OfflineIMAP; will detect
          this condition and abort prior to synchronization.)
        </para>
        <para>
          This question comes up frequently on the
          <ulink
          url="http://lists.complete.org/offlineimap@complete.org/">&OfflineIMAP;
          mailing list</ulink>.  You can find a
          <ulink
          url="http://lists.complete.org/offlineimap@complete.org/2003/04/msg00012.html.gz">detailed
          discussion</ulink> of the problem there.
        </para>
      </refsect2>

      <recsect2>
        <title>Use with MS Exchange server</title>
        <para>
          Several users have reported problems with Microsoft Exchange
          servers in conjunction with OfflineIMAP.  This generally
          seems to be related to the Exchange servers not properly
          following the IMAP standards.
        </para>
        <para>
          Mark Biggers has posted some <ulink
          url="http://lists.complete.org/offlineimap@complete.org/2005/09/msg00011.html.gz">information</ulink>
          to the &OfflineIMAP; mailing list about how he made it work.
        </para>
        <para>
          Other users have indicated that older (5.5) releases of
          Exchange are so bad that they will likely not work at all.
        </para>
        <para>
          I do not have access to Exchange servers for testing, so any
          problems with it, if they can even be solved at all, will
          require help from &OfflineIMAP; users to find and fix.
        </para>
      </recsect2>
          
    </refsect1>
    <refsect1>
      <title>Other Frequently Asked Questions</title>
      <para>There are some other FAQs that might not fit into another section
        of the document, so they are discussed here.
      </para>

      <variablelist>
        <varlistentry><term>What platforms does &OfflineIMAP; run on?</term>
          <listitem><para>
            It should run on most platforms supported by Python, which are quite a
            few.  I do not support Windows myself, but some have made
          it work there; see the FAQ entry for that platform.
          </para></listitem>
        </varlistentry>

        <varlistentry><term>I'm using Mutt.  Other IMAP sync programs require me to use "set maildir_trash=yes".  Do I need to do that with &OfflineIMAP;?</term>
          <listitem><para>
            No.  &OfflineIMAP; is smart enough to figure out message deletion without this extra
            crutch.  You'll get the best results if you don't use this setting, in
            fact.
          </para></listitem>
        </varlistentry>

        <varlistentry><term>I've upgraded and now &OfflineIMAP;
        crashes when I start it up!  Why?</term>
          <listitem><para>You need to upgrade your configuration
        file.  See <xref linkend="upgrading.4.0"> at the end of this
              manual.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry><term>How do I specify the names of my folders?</term>
          <listitem><para>
            You do not need to.  &OfflineIMAP; is smart
            enough to automatically figure out what folders are present
            on the IMAP server and synchronize them.  You can use the
            <property>folderfilter</property> and <property>nametrans</property>
            configuration file options to request certain folders and rename them
            as they come in if you like.
          </para></listitem>
        </varlistentry>

        <varlistentry><term>How can I prevent certain folders from being synced?</term>
          <listitem><para>
            Use the <property>folderfilter</property> option in the configuration file.
          </para></listitem>
        </varlistentry>

        <varlistentry><term>How can I add or delete a folder?</term>
          <listitem><para>
            &OfflineIMAP; does not currently provide this feature, but if you create a new
            folder on the IMAP server, it will be created locally automatically.
          </para></listitem>
        </varlistentry>

        <varlistentry><term>Are there any other warnings that I should be aware of?</term>
          <listitem><para>
            Yes; see the Notes section below.
          </para></listitem>
        </varlistentry>

        <varlistentry><term>What is the mailbox name recorder (mbnames) for?</term>
          <listitem><para>Some mail readers, such as Mutt, are not capable
            of automatically determining the names of your mailboxes.
            &OfflineIMAP; can help these programs by writing the names
            of the folders in a format you specify.  See the example
            <filename>offlineimap.conf</filename> for details.
          </para></listitem>
        </varlistentry>

        <varlistentry><term>Can I synchronize multiple accounts with &OfflineIMAP?</term>
          <listitem><para>Sure.  Just name them all in the
            <property>accounts</property> line in the <property>general</property>
            section of the configuration file, and add a per-account section
            for each one.
          </para></listitem>
        </varlistentry>

        <varlistentry><term>Does &OfflineIMAP; support POP?</term>
          <listitem><para>No.  POP is not robust enough to do a completely reliable
            multi-machine synchronization like &OfflineIMAP; can do.  &OfflineIMAP;
            will not support it.
          </para></listitem>
        </varlistentry>

        <varlistentry><term>Does &OfflineIMAP; support mailbox formats other than Maildir?</term>
          <listitem><para>Not at present.  There is no technical reason not to; just no
            demand yet.  Maildir is a superior format anyway.
              However, &OfflineIMAP; can sync between two IMAP
          servers, and some IMAP servers support other formats.  You
          could install an IMAP server on your local machine and have
          &OfflineIMAP sync to that.
          </para></listitem>
        </varlistentry>

        <varlistentry><term>[technical] Why are your Maildir message filenames so huge?</term>
          <listitem><para>&OfflineIMAP; has two relevant principles: 1) never modifying your
            messages in any way and 2) ensuring 100% reliable synchronizations.
            In order to do a reliable sync, &OfflineIMAP;
            must have a way to
            uniquely identify each e-mail.  Three pieces of information are
            required to do this: your account name, the folder name, and the
            message UID.  The account name can be calculated from the path in
            which your messages are.  The folder name can usually be as well, BUT
            some mail clients move messages between folders by simply moving the
            file, leaving the name intact.
          </para>
          <para>
            So, &OfflineIMAP; must store both a UID folder ID.  The folder ID is
            necessary so &OfflineIMAP; can detect a message moved to a different
            folder.  &OfflineIMAP; stores the UID (U= number) and an md5sum of the
            foldername (FMD5= number) to facilitate this.
          </para></listitem>
        </varlistentry>

        <varlistentry><term>What is the speed of &OfflineIMAP;'s sync?</term>
          <listitem><para>OfflineIMAP
            versions 2.0 and above contain a multithreaded system.  A good way to
            experiment is by setting <property>maxsyncaccounts</property> to 3 and <property>maxconnections</property> to 3
            in each account clause.
          </para>
          <para>This lets OfflineIMAP open up multiple connections simultaneously.
            That will let it process multiple folders and messages at once.  In
            most cases, this will increase performance of the sync.
          </para>
          <para>Don't set the number too high.  If you do that, things might actually
            slow down as your link gets saturated.  Also, too many connections can
            cause mail servers to have excessive load.  Administrators might take
            unkindly to this, and the server might bog down.  There are many
            variables in the optimal setting; experimentation may help.
          </para>
          <para>An informal benchmark yields these results for my setup:
          </para>
          <itemizedlist>
            <listitem><para>10 minutes with MacOS X Mail.app "manual cache"
              </para></listitem>
            <listitem><para>5 minutes with GNUS agent sync</para></listitem>
            <listitem><para>20 seconds with OfflineIMAP 1.x</para></listitem>
            <listitem><para>9 seconds with OfflineIMAP 2.x</para></listitem>
            <listitem><para>3 seconds with OfflineIMAP 3.x "cold start"</para></listitem>
            <listitem><para>2 seconds with OfflineIMAP 3.x "held connection"</para></listitem>
          </itemizedlist>
        </listitem></varlistentry>
        <varlistentry><term>Can I use &OfflineIMAP; on Windows?</term>
          <listitem><para>
              These answers have been reported by &OfflineIMAP;
              users.  I do not run &OfflineIMAP; on Windows myself, so
              I can't directly address their accuracy.
            </para>
            <para>
              The basic answer is that it's possible and doesn't
              require hacking &OfflineIMAP; source code.  However,
              it's not necessarily trivial.  The information below is
              based in instructions submitted by Chris Walker.
            </para>
            <para>
              First, you must run &OfflineIMAP; in the <ulink
              url="http://www.cygwin.com/">Cygwin</ulink>
              environment.  
            </para>
            <para>
              Next, you'll need to mount your Maildir directory in a
              special way.  There is information for doing that at
              <ulink url="http://barnson.org/node/view/295"></ulink>.
              That site gives this example:
            </para>
            <programlisting>
mount -f -s -b -o managed "d:/tmp/mail" "/home/of/mail"
            </programlisting> 
            <para>
              That URL also has more details on making OfflineIMAP
              work with Windows.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect1>

    <refsect1>
      <title>Conforming To</title>
      <itemizedlist>
        <listitem><para>Internet Message Access Protocol version 4rev1 (IMAP 4rev1) as
          specified in RFC2060 and RFC3501</para></listitem>
        <listitem><para>CRAM-MD5 as specified in RFC2195</para></listitem>
        <listitem><para>Maildir as specified in
          <ulink url="http://www.qmail.org/qmail-manual-html/man5/maildir.html">the Maildir manpage</ulink> and
          <ulink url="http://cr.yp.to/proto/maildir.html">the qmail website</ulink>.</para></listitem>
        <listitem><para>Standard Python 2.2.1 as implemented on POSIX-compliant systems.</para></listitem>
      </itemizedlist>
    </refsect1>

    <refsect1>
      <title>Notes</title>
      <refsect2>
        <title>Deleting Local Folders</title>
        <para>&OfflineIMAP; does a two-way synchronization.  That is, if you
          make a change to the mail on the server, it will be propagated to your
          local copy, and vise-versa.  Some people might think that it would be
          wise to just delete all their local mail folders periodically.  If you
          do this with &OfflineIMAP;, remember to also remove your local status
          cache (<filename>~/.offlineimap</filename> by default).  Otherwise, &OfflineIMAP; will take
          this as an intentional deletion of many messages and will interpret
          your action as requesting them to be deleted from the server as well.
          (If you don't understand this, don't worry; you probably won't
          encounter this situation)
        </para>
      </refsect2>

      <refsect2>
        <title>Multiple Instances</title>
        <para>&OfflineIMAP; is not designed to have several instances (for instance, a cron job and an interactive invocation) run over the same
          mailbox simultaneously.  It will perform a check on startup and
          abort if another &OfflineIMAP; is already running.  If you need
          to schedule synchronizations, please use the
          <property>autorefresh</property> settings rather than cron.
          Alternatively, you can set a separate <property>metadata</property>
          directory for each instance.
        </para>
      </refsect2>

      <refsect2>
        <title>Copying Messages Between Folders</title>
        <para>
          Normally, when you copy a message between folders or add a new message
          to a folder locally, &OfflineIMAP;
          will just do the right thing.  However, sometimes this can be tricky
          -- if your IMAP server does not provide the SEARCH command, or does
          not return something useful, &OfflineIMAP;
          cannot determine the new UID of the message.  So, in these rare
          instances, OfflineIMAP will upload the message to the IMAP server and
          delete it from your local folder.  Then, on your next sync, the
          message will be re-downloaded with the proper UID.
          &OfflineIMAP; makes sure that the message was properly uploaded before deleting it,
          so there should be no risk of data loss.
        </para>
      </refsect2>

      <refsect2>
        <title>Use with Evolution</title>
        <para>&OfflineIMAP; can work with Evolution.  To do so, first configure
          your &OfflineIMAP; account to have
          <option>sep = /</option> in its configuration.  Then, configure
          Evolution with the
          "Maildir-format mail directories" server type.  For the path, you will need to
          specify the name of the top-level folder
          <emphasis>inside</emphasis> your &OfflineIMAP; storage location.
          You're now set!
        </para>
      </refsect2>

      <refsect2>
        <title>Use with KMail</title>
        <para>At this time, I believe that &OfflineIMAP; with Maildirs
          is not compatible
          with KMail.  KMail cannot work in any mode other than to move
          all messages out of all folders immediately, which (besides being annoying
          and fundamentally broken) is incompatible with
        &OfflineIMAP;.
        </para>
        <para>
          However, I have made KMail version 3 work well with
          &OfflineIMAP; by installing an IMAP server on my local
          machine, having &OfflineIMAP; sync to that, and pointing
          KMail at the same server.
        </para>
      </refsect2>

      <refsect2>
        <title>Mailing List</title>
        <para>There is an OfflineIMAP mailing list available.
          To subscribe, send the text "Subscribe" in the subject of a mail to
          offlineimap-request@complete.org.  To post, send the message to
          offlineimap@complete.org.  Archives are available at
          <ulink url="http://lists.complete.org/offlineimap@complete.org/"></>.
        </para>
      </refsect2>

      <refsect2>
        <title>Bugs</title>
        <para>Reports of bugs should be sent via e-mail to the
          &OfflineIMAP; mailing list at offlineimap at complete
          dot org.  Debian users are encouraged to instead use the
        Debian
          bug-tracking system.
        </para>
      </refsect2>
    </refsect1>

    <refsect1 id="upgrading.4.0">
      <title>Upgrading to 4.0</title>
      <para>
        If you are upgrading from a version of &OfflineIMAP; prior to
        3.99.12, you will find that you will get errors when
        &OfflineIMAP; starts up (relating to ConfigParser or
        AccountHashGenerator) and the
        configuration file.  This is because the config file format
        had to change to accommodate new features in 4.0.  Fortunately,
        it's not difficult to adjust it to suit.
      </para>
      <para>
        First thing you need to do is stop any running &OfflineIMAP;
        instance, making sure first that it's synced all your mail.
        Then, modify your
        <filename>~/.offlineimaprc</filename> file.  You'll need to
        split up each account section (make sure that it now starts
        with "Account ") into two Repository sections (one for the
        local side and another for the remote side.)  See the files
        <filename>offlineimap.conf.minimal</filename> and
        <filename>offlineimap.conf</filename> in the distribution if
        you need more assistance.
      </para>
      <para>
        &OfflineIMAP;'s status directory area has also changed.
        Therefore, you should delete everything in ~/.offlineimap as
        well as your local mail folders.
      </para>
      <para>
        When you start up &OfflineIMAP; 4.0, it will re-download all
        your mail from the server and then you can continue using it
        like normal.
      </para>
    </refsect1>
        

    <refsect1>
      <title>Copyright</title>
      <para>OfflineIMAP, and this manual, are Copyright &copy; 2002, 2003 John Goerzen.</para>

      <para>
        This program is free software; you can redistribute it and/or modify
        it under the terms of the GNU General Public License as published by
        the Free Software Foundation; either version 2 of the License, or
        (at your option) any later version.
      </para>

      <para>
        This program is distributed in the hope that it will be useful,
        but WITHOUT ANY WARRANTY; without even the implied warranty of
        MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
        GNU General Public License for more details.
      </para>

      <para>
        You should have received a copy of the GNU General Public License
        along with this program; if not, write to the Free Software
        Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307 USA</para>

      <para>imaplib.py comes from the Python dev tree and is licensed under
        the GPL-compatible PSF license as stated in the file
        <filename>COPYRIGHT</filename> in the &OfflineIMAP;
         distribution.
      </para>
    </refsect1>

    <refsect1>
      <title>Author</title>
      <para>&OfflineIMAP;, its libraries, documentation, and all included files, except where
        noted, was written by John Goerzen <email>jgoerzen@complete.org</email> and
        copyright is held as stated in the COPYRIGHT section.
      </para>

      <para>
        &OfflineIMAP; may be downloaded, and information found, from its
        homepage via either <ulink url="gopher://quux.org/1/devel/offlineimap">Gopher</ulink>
        or <ulink url="http://quux.org/devel/offlineimap">HTTP</ulink>.
      </para>

      <para>
        &OfflineIMAP; may also be downloaded using Subversion.  Additionally,
        the distributed tar.gz may be updated with a simple "svn update"
        command; it is ready to go.  For information on getting OfflineIMAP
        with Subversion, please visit the
        <ulink url="http://svn.complete.org/">complete.org Subversion page</ulink>.
      </para>

    </refsect1>

    <refsect1>
      <title>See Also</title>
      <para><application>mutt</application>(1),
        <application>python</application>(1)
      </para>
    </refsect1>

    <refsect1>
      <title>History</title>
      <para>
        Detailed history may be found in the file ChangeLog in the
        &OfflineIMAP; distribution.  Feature and bug histories may be
        found in the file debian/changelog which, despite its name, is
        not really Debian-specific.  This section provides a large
        overview.
      </para>
      <para>
        Development on &OfflineIMAP; began on June 18, 2002.  Version
        1.0.0 was released three days later on June 21, 2002.  Point
        releases followed, including speed optimizations and some
        compatibility fixes.
      </para>
      <para>Version 2.0.0 was released on July 3, 2002, and
        represented the first time the synchronization became
        multithreaded and, to the best of my knowledge, the first
        multithreaded IMAP syncrhonizing application in existance.
        The last 2.0.x release, 2.0.8, was made on July 9.
      </para>
      <para>
        Version 3.0.0 was released on July 11, 2002, and introduced
        modular user interfaces and the first GUI interface for
        &OfflineIMAP;.  This manual also was introduced with 3.0.0,
        along with many command-line options.  Version 3.1.0 was
        released on July 21, adding the Noninteractive user
        interfaces, profiling support, and several bugfixes.  3.2.0
        was released on July 24, adding support for the Blinkenlights
        GUI interface.  &OfflineIMAP; entered maintenance mode for
        awhile, as it had reached a feature-complete milestone in my
        mind.
      </para>
      <para>
        The 3.99.x branch began in on October 7, 2002, to begin work
        for 4.0.  The Curses.Blinkenlights interface was added in
        3.99.6, and many architectural changes were made.
      </para>
      <para>
        4.0.0 was released on July 18, 2003, including the ability to
            synchronize directly between two IMAP servers, the first
            re-architecting of the configuration file to refine the
            notion of an account, and the new Curses interface.
      </para>
    </refsect1>
  </refentry>
</reference>

<!--
Local Variables:
mode: sgml
sgml-set-face: T
End:
-->