From: John Goerzen Date: Mon, 3 Mar 2008 08:27:13 +0000 (-0600) Subject: Merge branch 'netrc-integration' X-Git-Tag: debian/5.99.5~7 X-Git-Url: https://code.delx.au/offlineimap/commitdiff_plain/7b4e651d1282a50a20614074f731dd22f3ec642f?hp=9b8720a4ec8d0c2302fb523c3ca6620670007904 Merge branch 'netrc-integration' Applies patches by bboisin to add netrc support Conflicts: offlineimap/repository/IMAP.py refs #14 --- diff --git a/COPYRIGHT b/COPYRIGHT index 9333130..61aae00 100644 --- a/COPYRIGHT +++ b/COPYRIGHT @@ -1,5 +1,5 @@ offlineimap Mail syncing software -Copyright (C) 2002 - 2006 John Goerzen +Copyright (C) 2002 - 2007 John Goerzen This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -15,57 +15,3 @@ Copyright (C) 2002 - 2006 John Goerzen along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA -ONLY imaplib.py is Copyright (c) 2001 Python Software Foundation; -All Rights Reserved - -imaplib.py comes from Python dev tree and is licensed under the -GPL-compatible PSF license as follows: - - -PSF LICENSE AGREEMENT FOR PYTHON 2.2 ------------------------------------- - -1. This LICENSE AGREEMENT is between the Python Software Foundation -("PSF"), and the Individual or Organization ("Licensee") accessing and -otherwise using Python 2.2 software in source or binary form and its -associated documentation. - -2. Subject to the terms and conditions of this License Agreement, PSF -hereby grants Licensee a nonexclusive, royalty-free, world-wide -license to reproduce, analyze, test, perform and/or display publicly, -prepare derivative works, distribute, and otherwise use Python 2.2 -alone or in any derivative version, provided, however, that PSF's -License Agreement and PSF's notice of copyright, i.e., "Copyright (c) -2001 Python Software Foundation; All Rights Reserved" are retained in -Python 2.2 alone or in any derivative version prepared by Licensee. - -3. In the event Licensee prepares a derivative work that is based on -or incorporates Python 2.2 or any part thereof, and wants to make -the derivative work available to others as provided herein, then -Licensee hereby agrees to include in any such work a brief summary of -the changes made to Python 2.2. - -4. PSF is making Python 2.2 available to Licensee on an "AS IS" -basis. PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR -IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND -DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS -FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 2.2 WILL NOT -INFRINGE ANY THIRD PARTY RIGHTS. - -5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON -2.2 FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS -A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 2.2, -OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. - -6. This License Agreement will automatically terminate upon a material -breach of its terms and conditions. - -7. Nothing in this License Agreement shall be deemed to create any -relationship of agency, partnership, or joint venture between PSF and -Licensee. This License Agreement does not grant permission to use PSF -trademarks or trade name in a trademark sense to endorse or promote -products or services of Licensee, or any third party. - -8. By copying, installing or otherwise using Python 2.2, Licensee -agrees to be bound by the terms and conditions of this License -Agreement. diff --git a/bin/offlineimap b/bin/offlineimap index 95fd2fb..226bf3c 100644 --- a/bin/offlineimap +++ b/bin/offlineimap @@ -1,6 +1,6 @@ #!/usr/bin/env python # Startup from system-wide installation -# Copyright (C) 2002 - 2006 John Goerzen +# Copyright (C) 2002 - 2007 John Goerzen # # # This program is free software; you can redistribute it and/or modify @@ -18,4 +18,4 @@ # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA from offlineimap import init -init.startup('4.0.16') +init.startup('5.99.4') diff --git a/debian/changelog b/debian/changelog index 06cbb14..41a3289 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,91 @@ +offlineimap (5.99.5) unstable; urgency=low + + * When a local folder is deleted, but the OfflineIMAP status cache + is not cleared, do not delete all mail in that folder on the + remote. Closes: #459985. + * Apply SSL performance enhancement, working around severe + inefficiencies in Python's imaplib.py. This workaround was lost + in 5.99.0 when the locally-hacked imaplib.py was removed. Thanks + to Aaron Kaplan for help isolating the problem. + * Added special support for GMail. Patch from Riccardo Murri. + * Added option 'f' for syncing only selected folders. Patch from + Riccardo Murri. + * Added option -k for overriding config options. Patch from + Riccardo Murri. + * Fix read-only detection behavior when changing flags. Patch from + Florian Friesdorf. + * Fix handling of servers that return UIDs in some, but not all, + responses to FETCH. OfflineIMAP bug #22. Patch by pistore. + * Fix typo in manual. Closes: #360365. + + -- John Goerzen Mon, 03 Mar 2008 00:42:56 -0600 + +offlineimap (5.99.4) unstable; urgency=low + + * Undo 'Fix Maildir race' patch. It was causes spinning. + Closes: #447217, #447226. + + -- John Goerzen Fri, 19 Oct 2007 06:31:09 -0500 + +offlineimap (5.99.3) unstable; urgency=low + + * Fix a potential Maildir race. Thanks to Martin F. Krafft for the + patch. Closes: #439384. + * SGML syntax fix. + * Patches from Daniel Jacobowitz: + + New -q (quick) option to only update folders that seem to have + significant changes + + Fix to UID validity problems some had reported with Courier + + Maildir checking speedups + + Display fixes for Curses.Blinkenlights + * Fixed locked( )for noninteractive UIs. Thanks to Mark A. Hershberger + for the patch. + * Configurable thread status character for ui.Curses.Blinkenlights. + Thanks to aaron@schrab.com for the patch. + * Documented remotehosteval. Thanks to Ben Kibbey. + * [debian] Added build-dep on man-db. Closes: #433739. + + -- John Goerzen Thu, 18 Oct 2007 14:03:39 -0500 + +offlineimap (5.99.2) unstable; urgency=low + + * Add build-dep on gs. Closes: #433739. + * Additional date check. Closes: #396443. + + -- John Goerzen Tue, 31 Jul 2007 05:27:05 -0500 + +offlineimap (5.99.1) unstable; urgency=low + + * Remove suggestion on python-tk. Closes: #432843. + * Fix non-SSL connection bug introduced in 5.99.0. + Thanks to Eloy Paris for pinpointing it. Closes: #432727. + + -- John Goerzen Thu, 12 Jul 2007 01:03:08 -0500 + +offlineimap (5.99.0) unstable; urgency=low + + * Re-scan remote folder names at the start of each sync run. + Closes: #329000, #396772. + * Drop internal imaplib.py in favor of default Python one. + * New user interface: Machine.MachineUI. Machine-parsable. + * Drop all the Tk interfaces. + Closes: #265088. + * Now supports specifying timeouts for socket operations. + * Updated copyright files. + * Improved interaction with Dovecot maildirs. Patch thanks to Asheesh + Laroia. + * Improved filesystem syncing semantics, which should reduce duplication + in the event of hardware failure. + * UID validity diagnostics improvement. Patch from David Favro. + * No longer leave preauthtunnel zombies with autorefresh. + Patch from Peter Colberg. Closes: #410730. + * --help now shows available UIs. Patch from Daniel Rall. + * Check all resolved addresses. Patch from Mark Brown. Closes: #413030. + * Removed todo directory from tree, moved to BTS. + * New PID file to enable third-party "kill offlineimap" tools. + + -- John Goerzen Tue, 10 Jul 2007 04:10:26 -0500 + offlineimap (4.0.16) unstable; urgency=low * Apply patches from Danial Burrows to improve situation when diff --git a/debian/control b/debian/control index 397993e..05732d5 100644 --- a/debian/control +++ b/debian/control @@ -3,14 +3,13 @@ Section: mail Priority: optional Maintainer: John Goerzen Build-Depends: debhelper (>> 4.2.0) -Build-Depends-Indep: python-dev, groff, docbook-utils, python-support (>= 0.4.0), gtk-doc-tools, sgml2x, docbook-utils, jade, lynx, scons, xpdf-utils +Build-Depends-Indep: python-dev, groff, docbook-utils, python-support (>= 0.4.0), gtk-doc-tools, sgml2x, docbook-utils, jade, lynx, scons, xpdf-utils, gs, man-db Build-Conflicts-Indep: docbook-ebnf Standards-Version: 3.7.2 Package: offlineimap Architecture: all Depends: ${python:Depends} -Suggests: python-tk Description: IMAP/Maildir synchronization and reader support OfflineIMAP is a tool to simplify your e-mail reading. With OfflineIMAP, you can: diff --git a/debian/copyright b/debian/copyright index 5635f7a..84f1b97 100644 --- a/debian/copyright +++ b/debian/copyright @@ -4,7 +4,7 @@ on Fri, 21 Jun 2002 14:54:56 -0500. The original source can always be found at: http://software.complete.org/offlineimap/ -Copyright (C) 2002 - 2006 John Goerzen +Copyright (C) 2002 - 2007 John Goerzen This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by diff --git a/offlineimap.conf b/offlineimap.conf index 1f15041..a5b15d1 100644 --- a/offlineimap.conf +++ b/offlineimap.conf @@ -51,18 +51,18 @@ maxsyncaccounts = 1 # fails, the second, and so forth. # # The pre-defined options are: -# Tk.Blinkenlights -- A graphical interface, shows LEDs and a single log -# Tk.VerboseUI -- A graphical interface, shows logs per thread # Curses.Blinkenlights -- A text-based (terminal) interface similar to # Tk.Blinkenlights # TTY.TTYUI -- a text-based (terminal) interface # Noninteractive.Basic -- Noninteractive interface suitable for cronning # Noninteractive.Quiet -- Noninteractive interface, generates no output # except for errors. +# Machine.MachineUI -- Interactive interface suitable for machine +# parsing. # # You can override this with a command-line option -u. -ui = Tk.Blinkenlights, Tk.VerboseUI, Curses.Blinkenlights, TTY.TTYUI, +ui = Curses.Blinkenlights, TTY.TTYUI, Noninteractive.Basic, Noninteractive.Quiet # If you try to synchronize messages to a read-only folder, @@ -85,6 +85,19 @@ ignore-readonly = no # pythonfile = ~/.offlineimap.py # +# By default, OfflineIMAP will not exit due to a network error until +# the operating system returns an error code. Operating systems can sometimes +# take forever to notice this. Here you can activate a timeout on the +# socket. This timeout applies to individual socket reads and writes, +# not to an overall sync operation. You could perfectly well have a 30s +# timeout here and your sync still take minutes. +# +# Values in the 30-120 second range are reasonable. +# +# The default is to have no timeout beyond the OS. Times are given in seconds. +# +# socktimeout = 60 + ################################################## # Mailbox name recorder ################################################## @@ -116,29 +129,10 @@ footer = "\n" # Note that this filter can be used only to further restrict mbnames # to a subset of folders that pass the account's folderfilter. -################################################## -# Blinkenlights configuration -################################################## - -[ui.Tk.Blinkenlights] - -# Specifies the default number of lines in the log. - -loglines = 5 - -# Specifies how many lines are in the scrollback log buffer. - -bufferlines = 500 - -# If true, says that the log should be enabled by default. -# Otherwise, you have to click "Show Log" to enable the log. - -showlog = false +[ui.Curses.Blinkenlights] +# Character used to indicate thread status. -# Sets the font information. - -fontfamily = Helvetica -fontsize = 8 +statuschar = . ################################################## # Accounts @@ -165,6 +159,16 @@ remoterepository = RemoteExample # autorefresh = 5 +# You can tell offlineimap to do a number of quicker synchronizations +# between full updates. A quick synchronization only synchronizes +# if a Maildir folder has changed, or if an IMAP folder has received +# new messages or had messages deleted. It does not update if the +# only changes were to IMAP flags. Specify 0 to never do quick updates, +# -1 to always do quick updates, or a positive integer to do that many +# quick updates between each full synchronization (requires autorefresh). + +# quick = 10 + [Repository LocalExample] # This is one of the two repositories that you'll work with given the @@ -199,10 +203,18 @@ restoreatime = no [Repository RemoteExample] -# And this is the remote repository. For now, we only support IMAP here. +# And this is the remote repository. We only support IMAP or Gmail here. type = IMAP +# The following can fetch the account credentials via a python expression that +# is parsed from the pythonfile parameter. For example, a function called +# "getcredentials" that parses a file "filename" and returns the account +# details for "hostname". +# remotehosteval = getcredentials("filename", "hostname", "hostname") +# remoteusereval = getcredentials("filename", "hostname", "user") +# remotepasseval = getcredentials("filename", "hostname", "passwd") + # Specify the remote hostname. remotehost = examplehost @@ -370,3 +382,33 @@ holdconnectionopen = no # # foldersort = lambda x, y: -cmp(x, y) + +[Repository GmailExample] + +# A repository using Gmail's IMAP interface. Any configuration +# parameter of `IMAP` type repositories can be used here. Only +# `remoteuser` (or `remoteusereval` ) is mandatory. Default values +# for other parameters are OK, and you should not need fiddle with +# those. +# +# The Gmail repository will use hard-coded values for `remotehost`, +# `remoteport`, `tunnel` and `ssl`. (See +# http://mail.google.com/support/bin/answer.py?answer=78799&topic=12814) +# Any attempt to set those parameters will be silently ignored. +# + +type = Gmail + +# Specify the Gmail user name. This is the only mandatory parameter. +remoteuser = username@gmail.com + +# Deleting a message from a Gmail folder via the IMAP interface will +# just remove that folder's label from the message: the message will +# continue to exist in the '[Gmail]/All Mail' folder. If `realdelete` +# is set to `True`, then deleted messages will really be deleted +# during `offlineimap` sync, by moving them to the '[Gmail]/Trash' +# folder. BEWARE: this will deleted a messages from *all folders* it +# belongs to! +# +# See http://mail.google.com/support/bin/answer.py?answer=77657&topic=12815 +realdelete = no diff --git a/offlineimap.py b/offlineimap.py index 32009ac..07a4aeb 100644 --- a/offlineimap.py +++ b/offlineimap.py @@ -1,6 +1,6 @@ #!/usr/bin/env python # Startup from single-user installation -# Copyright (C) 2002 - 2006 John Goerzen +# Copyright (C) 2002 - 2007 John Goerzen # # # This program is free software; you can redistribute it and/or modify @@ -18,4 +18,4 @@ # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA from offlineimap import init -init.startup('4.0.16') +init.startup('5.99.4') diff --git a/offlineimap.sgml b/offlineimap.sgml index 0bd0ce3..83f4ac0 100644 --- a/offlineimap.sgml +++ b/offlineimap.sgml @@ -32,6 +32,8 @@ -a accountlist -c configfile -d debugtype[,...] + -f foldername[,...] + -k [section:]option=value -l filename -o -u interface @@ -204,18 +206,19 @@ remoteuser = jgoerzen and corporate networks do, and most operating systems have an IMAP implementation readily available. + A special Gmail mailbox type is + available to interface with Gmail's IMAP front-end. - You must have Python version 2.2.1 or above installed. + You must have Python version 2.4 or above installed. If you are running on Debian GNU/Linux, this requirement will automatically be taken care of for you. If you do not have Python already, check with your system administrator or operating system vendor; or, download it from the Python website. - If you intend to use the Tk interface, you must have Tkinter - (python-tk) installed. If you intend to use the SSL interface, your + If you intend to use the SSL interface, your Python must have been built with SSL support. @@ -231,6 +234,7 @@ remoteuser = jgoerzen IMAP server and point both &OfflineIMAP; and your mail reader at it. + @@ -373,6 +377,26 @@ cd offlineimap-x.y.z will debug the threading model. + -f foldername[,foldername] + Only sync the specified folders. The + foldernames are the + untranslated foldernames. This command-line option + overrides any folderfilter + and folderincludes options in the + configuration file. + + + -k [section:]option=value + + Override configuration file option. If + "section" is omitted, it defaults + to general. Any underscores "_" in + the section name are replaced with spaces: for instance, + to override option autorefresh in + the "[Account Personal]" section in the config file one + would use "-k Account_Personal:autorefresh=30". + + -l filename @@ -389,6 +413,11 @@ cd offlineimap-x.y.z file. + -q + Run only quick synchronizations. Ignore any flag + updates on IMAP servers. + + -h --help Show summary of options. @@ -412,27 +441,22 @@ cd offlineimap-x.y.z option can override the configuration file setting. The available values for the configuration file or command-line are described in this section. + - Tk.Blinkenlights - Tk.Blinkenlights is an interface designed to be sleek, fun to watch, and + Curses.Blinkenlights + + Curses.Blinkenlights is an interface designed to be sleek, fun to watch, and informative of the overall picture of what &OfflineIMAP; is doing. I consider it to be the best general-purpose interface in &OfflineIMAP;. - Tk.Blinkenlights contains, by default, a small window with a row of - LEDs, a small log, and a row of command buttons. - The total size of the window is - very small, so it uses little desktop space, yet it is quite - functional. The optional, toggleable, log shows more + Curses.Blinkenlights contains a row of + "LEDs" with command buttons and a log. + The log shows more detail about what is happening and is color-coded to match the color of the lights. - - Tk.Blinkenlights is the only user interface that has configurable - parameters; see the example offlineimap.conf - for more details. - Each light in the Blinkenlights interface represents a thread of execution -- that is, a particular task that &OfflineIMAP; @@ -538,32 +562,6 @@ cd offlineimap-x.y.z - - Curses.Blinkenlights - - Curses.Blinkenlights is an interface very similar to Tk.Blinkenlights, - but is designed to be run in a console window (an xterm, Linux virtual - terminal, etc.) Since it doesn't have access to graphics, it isn't - quite as pretty, but it still gets the job done. - - Please see the Tk.Blinkenlights section above for more - information about the colors used in this interface. - - - - - Tk.VerboseUI - - Tk.VerboseUI (formerly known as Tk.TkUI) is a graphical interface - that presents a variable-sized window. In the window, each - currently-executing thread has a section where its name and current - status are displayed. This interface is best suited to people running - on slower connections, as you get a lot of detail, but for fast - connections, the detail may go by too quickly to be useful. People - with fast connections may wish to use Tk.Blinkenlights instead. - - - TTY.TTYUI @@ -598,6 +596,15 @@ cd offlineimap-x.y.z + + Machine.MachineUI + + Machine.MachineUI generates output in a machine-parsable format. + It is designed for other programs that will interface + to OfflineIMAP. + + + @@ -694,7 +701,7 @@ holdconnectionopen = no is based on one supplied by Tommi Virtanen for this feature. - In ~/.offlineimap.rc, he adds these options: + In ~/.offlineimaprc, he adds these options: [general] pythonfile=~/.offlineimap.py @@ -731,6 +738,24 @@ def test_mycmp(): + + + Signals + + OfflineIMAP writes its current PID into + ~/.offlineimap/pid when it is + running. It is not guaranteed that this file will + not exist when OfflineIMAP is not running. + + + Errors @@ -766,11 +791,11 @@ def test_mycmp(): rm -r ~/Folders/INBOX -rm -r ~/.offlineimap/Account-AccountName -rm -r ~/.offlineimap/Repository-RepositoryName +rm -r ~/.offlineimap/Account-AccountName/LocalStatus/INBOX +rm -r ~/.offlineimap/Repository-RemoteRepositoryName/FolderValidity/INBOX - (Of course, replace AccountName and RepositoryName + (Of course, replace AccountName and RemoteRepositoryName with the names as specified in ~/.offlineimaprc). @@ -836,8 +861,8 @@ rm -r ~/.offlineimap/Repository-RepositoryName &OfflineIMAP; is not designed to have several instances (for instance, a cron job and an interactive invocation) run over the same mailbox simultaneously. It will perform a check on startup and abort if another &OfflineIMAP; is already running. If you need - to schedule synchronizations, please use the - autorefresh settings rather than cron. + to schedule synchronizations, you'll probably find + autorefresh settings more convenient than cron. Alternatively, you can set a separate metadata directory for each instance. diff --git a/offlineimap/accounts.py b/offlineimap/accounts.py index 0047e1f..8e96347 100644 --- a/offlineimap/accounts.py +++ b/offlineimap/accounts.py @@ -45,6 +45,7 @@ class Account(CustomConfig.ConfigHelperMixin): self.localeval = config.getlocaleval() self.ui = UIBase.getglobalui() self.refreshperiod = self.getconffloat('autorefresh', 0.0) + self.quicknum = 0 if self.refreshperiod == 0.0: self.refreshperiod = None @@ -125,12 +126,26 @@ class AccountSynchronizationMixin: def sync(self): # We don't need an account lock because syncitall() goes through # each account once, then waits for all to finish. + + quickconfig = self.getconfint('quick', 0) + if quickconfig < 0: + quick = True + elif quickconfig > 0: + if self.quicknum == 0 or self.quicknum > quickconfig: + self.quicknum = 1 + quick = False + else: + self.quicknum = self.quicknum + 1 + quick = True + else: + quick = False + try: remoterepos = self.remoterepos localrepos = self.localrepos statusrepos = self.statusrepos self.ui.syncfolders(remoterepos, localrepos) - remoterepos.syncfoldersto(localrepos) + remoterepos.syncfoldersto(localrepos, [statusrepos]) folderthreads = [] for remotefolder in remoterepos.getfolders(): @@ -140,12 +155,14 @@ class AccountSynchronizationMixin: name = "Folder sync %s[%s]" % \ (self.name, remotefolder.getvisiblename()), args = (self.name, remoterepos, remotefolder, localrepos, - statusrepos)) + statusrepos, quick)) thread.setDaemon(1) thread.start() folderthreads.append(thread) threadutil.threadsreset(folderthreads) mbnames.write() + localrepos.forgetfolders() + remoterepos.forgetfolders() localrepos.holdordropconnections() remoterepos.holdordropconnections() finally: @@ -155,7 +172,7 @@ class SyncableAccount(Account, AccountSynchronizationMixin): pass def syncfolder(accountname, remoterepos, remotefolder, localrepos, - statusrepos): + statusrepos, quick): global mailboxes ui = UIBase.getglobalui() ui.registerthread(accountname) @@ -165,12 +182,6 @@ def syncfolder(accountname, remoterepos, remotefolder, localrepos, replace(remoterepos.getsep(), localrepos.getsep())) # Write the mailboxes mbnames.add(accountname, localfolder.getvisiblename()) - # Load local folder - ui.syncingfolder(remoterepos, remotefolder, localrepos, localfolder) - ui.loadmessagelist(localrepos, localfolder) - localfolder.cachemessagelist() - ui.messagelistloaded(localrepos, localfolder, len(localfolder.getmessagelist().keys())) - # Load status folder. statusfolder = statusrepos.getfolder(remotefolder.getvisiblename().\ @@ -183,6 +194,19 @@ def syncfolder(accountname, remoterepos, remotefolder, localrepos, statusfolder.cachemessagelist() + if quick: + if not localfolder.quickchanged(statusfolder) \ + and not remotefolder.quickchanged(statusfolder): + ui.skippingfolder(remotefolder) + localrepos.restore_atime() + return + + # Load local folder + ui.syncingfolder(remoterepos, remotefolder, localrepos, localfolder) + ui.loadmessagelist(localrepos, localfolder) + localfolder.cachemessagelist() + ui.messagelistloaded(localrepos, localfolder, len(localfolder.getmessagelist().keys())) + # If either the local or the status folder has messages and there is a UID # validity problem, warn and abort. If there are no messages, UW IMAPd # loses UIDVALIDITY. But we don't really need it if both local folders are diff --git a/offlineimap/folder/Gmail.py b/offlineimap/folder/Gmail.py new file mode 100644 index 0000000..bf040e9 --- /dev/null +++ b/offlineimap/folder/Gmail.py @@ -0,0 +1,119 @@ +# Gmail IMAP folder support +# Copyright (C) 2008 Riccardo Murri +# Copyright (C) 2002-2007 John Goerzen +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + +"""Folder implementation to support features of the Gmail IMAP server. +""" + +from IMAP import IMAPFolder +import imaplib +from offlineimap import imaputil, imaplibutil +from offlineimap.ui import UIBase +from copy import copy + + +class GmailFolder(IMAPFolder): + """Folder implementation to support features of the Gmail IMAP server. + Specifically, deleted messages are moved to folder `Gmail.TRASH_FOLDER` + (by default: ``[Gmail]/Trash``) prior to expunging them, since + Gmail maps to IMAP ``EXPUNGE`` command to "remove label". + + For more information on the Gmail IMAP server: + http://mail.google.com/support/bin/answer.py?answer=77657&topic=12815 + """ + + #: Where deleted mail should be moved + TRASH_FOLDER='[Gmail]/Trash' + + #: Gmail will really delete messages upon EXPUNGE in these folders + REAL_DELETE_FOLDERS = [ TRASH_FOLDER, '[Gmail]/Spam' ] + + def __init__(self, imapserver, name, visiblename, accountname, repository): + self.realdelete = repository.getrealdelete(name) + IMAPFolder.__init__(self, imapserver, name, visiblename, \ + accountname, repository) + + def deletemessages_noconvert(self, uidlist): + uidlist = [uid for uid in uidlist if uid in self.messagelist] + if not len(uidlist): + return + + if self.realdelete and not (self.getname() in self.REAL_DELETE_FOLDERS): + # IMAP expunge is just "remove label" in this folder, + # so map the request into a "move into Trash" + + imapobj = self.imapserver.acquireconnection() + try: + imapobj.select(self.getfullname()) + result = imapobj.uid('copy', + imaputil.listjoin(uidlist), + self.TRASH_FOLDER) + assert result[0] == 'OK', \ + "Bad IMAPlib result: %s" % result[0] + finally: + self.imapserver.releaseconnection(imapobj) + for uid in uidlist: + del self.messagelist[uid] + else: + IMAPFolder.deletemessages_noconvert(self, uidlist) + + def processmessagesflags(self, operation, uidlist, flags): + # XXX: the imapobj.myrights(...) calls dies with an error + # report from Gmail server stating that IMAP command + # 'MYRIGHTS' is not implemented. So, this + # `processmessagesflags` is just a copy from `IMAPFolder`, + # with the references to `imapobj.myrights()` deleted This + # shouldn't hurt, however, Gmail users always have full + # control over all their mailboxes (apparently). + if len(uidlist) > 101: + # Hack for those IMAP ervers with a limited line length + self.processmessagesflags(operation, uidlist[:100], flags) + self.processmessagesflags(operation, uidlist[100:], flags) + return + + imapobj = self.imapserver.acquireconnection() + try: + imapobj.select(self.getfullname()) + r = imapobj.uid('store', + imaputil.listjoin(uidlist), + operation + 'FLAGS', + imaputil.flagsmaildir2imap(flags)) + assert r[0] == 'OK', 'Error with store: ' + '. '.join(r[1]) + r = r[1] + finally: + self.imapserver.releaseconnection(imapobj) + + needupdate = copy(uidlist) + for result in r: + attributehash = imaputil.flags2hash(imaputil.imapsplit(result)[1]) + flags = attributehash['FLAGS'] + uid = long(attributehash['UID']) + self.messagelist[uid]['flags'] = imaputil.flagsimap2maildir(flags) + try: + needupdate.remove(uid) + except ValueError: # Let it slide if it's not in the list + pass + for uid in needupdate: + if operation == '+': + for flag in flags: + if not flag in self.messagelist[uid]['flags']: + self.messagelist[uid]['flags'].append(flag) + self.messagelist[uid]['flags'].sort() + elif operation == '-': + for flag in flags: + if flag in self.messagelist[uid]['flags']: + self.messagelist[uid]['flags'].remove(flag) diff --git a/offlineimap/folder/IMAP.py b/offlineimap/folder/IMAP.py index d890596..6b0e805 100644 --- a/offlineimap/folder/IMAP.py +++ b/offlineimap/folder/IMAP.py @@ -1,5 +1,5 @@ # IMAP folder support -# Copyright (C) 2002-2004 John Goerzen +# Copyright (C) 2002-2007 John Goerzen # # # This program is free software; you can redistribute it and/or modify @@ -17,7 +17,8 @@ # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA from Base import BaseFolder -from offlineimap import imaputil, imaplib +import imaplib +from offlineimap import imaputil, imaplibutil from offlineimap.ui import UIBase from offlineimap.version import versionstr import rfc822, time, string, random, binascii, re @@ -40,6 +41,16 @@ class IMAPFolder(BaseFolder): self.randomgenerator = random.Random() BaseFolder.__init__(self) + def selectro(self, imapobj): + """Select this folder when we do not need write access. + Prefer SELECT to EXAMINE if we can, since some servers + (Courier) do not stabilize UID validity until the folder is + selected.""" + try: + imapobj.select(self.getfullname()) + except imapobj.readonly: + imapobj.select(self.getfullname(), readonly = 1) + def getaccountname(self): return self.accountname @@ -59,11 +70,52 @@ class IMAPFolder(BaseFolder): imapobj = self.imapserver.acquireconnection() try: # Primes untagged_responses - imapobj.select(self.getfullname(), readonly = 1) + self.selectro(imapobj) return long(imapobj.untagged_responses['UIDVALIDITY'][0]) finally: self.imapserver.releaseconnection(imapobj) + def quickchanged(self, statusfolder): + # An IMAP folder has definitely changed if the number of + # messages or the UID of the last message have changed. Otherwise + # only flag changes could have occurred. + imapobj = self.imapserver.acquireconnection() + try: + # Primes untagged_responses + imapobj.select(self.getfullname(), readonly = 1, force = 1) + try: + # Some mail servers do not return an EXISTS response if + # the folder is empty. + maxmsgid = long(imapobj.untagged_responses['EXISTS'][0]) + except KeyError: + return True + + # Different number of messages than last time? + if maxmsgid != len(statusfolder.getmessagelist()): + return True + + if maxmsgid < 1: + # No messages; return + return False + + # Now, get the UID for the last message. + response = imapobj.fetch('%d' % maxmsgid, '(UID)')[1] + finally: + self.imapserver.releaseconnection(imapobj) + + # Discard the message number. + messagestr = string.split(response[0], maxsplit = 1)[1] + options = imaputil.flags2hash(messagestr) + if not options.has_key('UID'): + return True + uid = long(options['UID']) + saveduids = statusfolder.getmessagelist().keys() + saveduids.sort() + if uid != saveduids[-1]: + return True + + return False + def cachemessagelist(self): imapobj = self.imapserver.acquireconnection() self.messagelist = {} @@ -98,7 +150,7 @@ class IMAPFolder(BaseFolder): else: uid = long(options['UID']) flags = imaputil.flagsimap2maildir(options['FLAGS']) - rtime = imaplib.Internaldate2epoch(messagestr) + rtime = imaplibutil.Internaldate2epoch(messagestr) self.messagelist[uid] = {'uid': uid, 'flags': flags, 'time': rtime} def getmessagelist(self): @@ -212,6 +264,12 @@ class IMAPFolder(BaseFolder): try: if datetuple[0] < 1981: raise ValueError + + # Check for invalid date + datetuple_check = time.localtime(time.mktime(datetuple)) + if datetuple[:2] != datetuple_check[:2]: + raise ValueError + # This could raise a value error if it's not a valid format. date = imaplib.Time2Internaldate(datetuple) except (ValueError, OverflowError): @@ -270,7 +328,7 @@ class IMAPFolder(BaseFolder): return result = imapobj.uid('store', '%d' % uid, 'FLAGS', imaputil.flagsmaildir2imap(flags)) - assert result[0] == 'OK', 'Error with store: ' + r[1] + assert result[0] == 'OK', 'Error with store: ' + '. '.join(r[1]) finally: self.imapserver.releaseconnection(imapobj) result = result[1][0] @@ -307,16 +365,35 @@ class IMAPFolder(BaseFolder): imapobj = self.imapserver.acquireconnection() try: + # Making sure, that we have the necessary rights + # ensuring that we access readonly: python's braindead imaplib.py + # otherwise might raise an exception during the myrights() call + imapobj.select(self.getfullname(),readonly=1) + myrights = imapobj.myrights(self.getfullname())[1][0].split()[1] + if 'T' in flags and not 'd' in myrights or \ + 'S' in flags and not 's' in myrights or \ + filter(lambda x: x not in 'TS', flags) and not 'w' in myrights: + # no delete/expunge right, but needed or + # no store seen right, but needed or + # no write right, but needed + UIBase.getglobalui().flagstoreadonly(self, uidlist, flags) + return + try: imapobj.select(self.getfullname()) except imapobj.readonly: - UIBase.getglobalui().flagstoreadonly(self, uidlist, flags) - return + # Above we made sure, we have the necessary rights. + # Ugly hack, to prevent an unnecessary exception: + # readonly: mailbox status changed to READ-ONLY + # imaplib should take care of that itself. + # The connection is anyway released below, so we dont need to + # undo the hack. + imapobj.is_readonly = True r = imapobj.uid('store', imaputil.listjoin(uidlist), operation + 'FLAGS', imaputil.flagsmaildir2imap(flags)) - assert r[0] == 'OK', 'Error with store: ' + r[1] + assert r[0] == 'OK', 'Error with store: ' + '. '.join(r[1]) r = r[1] finally: self.imapserver.releaseconnection(imapobj) @@ -333,9 +410,9 @@ class IMAPFolder(BaseFolder): if not ('UID' in attributehash and 'FLAGS' in attributehash): # Compensate for servers that don't return a UID attribute. continue - flags = attributehash['FLAGS'] + lflags = attributehash['FLAGS'] uid = long(attributehash['UID']) - self.messagelist[uid]['flags'] = imaputil.flagsimap2maildir(flags) + self.messagelist[uid]['flags'] = imaputil.flagsimap2maildir(lflags) try: needupdate.remove(uid) except ValueError: # Let it slide if it's not in the list @@ -366,12 +443,17 @@ class IMAPFolder(BaseFolder): self.addmessagesflags_noconvert(uidlist, ['T']) imapobj = self.imapserver.acquireconnection() try: - try: - imapobj.select(self.getfullname()) - except imapobj.readonly: + # Making sure, that we have the necessary rights + # ensuring that we access readonly: python's braindead imaplib.py + # otherwise might raise an exception during the myrights() call + imapobj.select(self.getfullname(),readonly=1) + if not 'd' in imapobj.myrights(self.getfullname())[1][0].split()[1]: + # no delete/expunge rights UIBase.getglobalui().deletereadonly(self, uidlist) return + if self.expunge: + imapobj.select(self.getfullname()) assert(imapobj.expunge()[0] == 'OK') finally: self.imapserver.releaseconnection(imapobj) diff --git a/offlineimap/folder/Maildir.py b/offlineimap/folder/Maildir.py index a4dbdba..56a63e4 100644 --- a/offlineimap/folder/Maildir.py +++ b/offlineimap/folder/Maildir.py @@ -22,7 +22,6 @@ from offlineimap.ui import UIBase from threading import Lock import os.path, os, re, time, socket, md5 -foldermatchre = re.compile(',FMD5=([0-9a-f]{32})') uidmatchre = re.compile(',U=(\d+)') flagmatchre = re.compile(':.*2,([A-Z]+)') @@ -78,16 +77,16 @@ class MaildirFolder(BaseFolder): files = [] nouidcounter = -1 # Messages without UIDs get # negative UID numbers. + foldermd5 = md5.new(self.getvisiblename()).hexdigest() + folderstr = ',FMD5=' + foldermd5 for dirannex in ['new', 'cur']: fulldirname = os.path.join(self.getfullname(), dirannex) files.extend([os.path.join(fulldirname, filename) for filename in os.listdir(fulldirname)]) for file in files: messagename = os.path.basename(file) - foldermatch = foldermatchre.search(messagename) - if (not foldermatch) or \ - md5.new(self.getvisiblename()).hexdigest() \ - != foldermatch.group(1): + foldermatch = messagename.find(folderstr) != -1 + if not foldermatch: # If there is no folder MD5 specified, or if it mismatches, # assume it is a foreign (new) message and generate a # negative uid for it @@ -111,8 +110,21 @@ class MaildirFolder(BaseFolder): 'filename': file} return retval + def quickchanged(self, statusfolder): + self.cachemessagelist() + savedmessages = statusfolder.getmessagelist() + if len(self.messagelist) != len(savedmessages): + return True + for uid in self.messagelist.keys(): + if uid not in savedmessages: + return True + if self.messagelist[uid]['flags'] != savedmessages[uid]['flags']: + return True + return False + def cachemessagelist(self): - self.messagelist = self._scanfolder() + if self.messagelist is None: + self.messagelist = self._scanfolder() def getmessagelist(self): return self.messagelist @@ -130,6 +142,8 @@ class MaildirFolder(BaseFolder): return st.st_mtime def savemessage(self, uid, content, flags, rtime): + # This function only ever saves to tmp/, + # but it calls savemessageflags() to actually save to cur/ or new/. ui = UIBase.getglobalui() ui.debug('maildir', 'savemessage: called to write with flags %s and content %s' % \ (repr(flags), repr(content))) @@ -140,12 +154,9 @@ class MaildirFolder(BaseFolder): # We already have it. self.savemessageflags(uid, flags) return uid - if 'S' in flags: - # If a message has been seen, it goes into the cur - # directory. CR debian#152482, [complete.org #4] - newdir = os.path.join(self.getfullname(), 'cur') - else: - newdir = os.path.join(self.getfullname(), 'new') + + # Otherwise, save the message in tmp/ and then call savemessageflags() + # to give it a permanent home. tmpdir = os.path.join(self.getfullname(), 'tmp') messagename = None attempts = 0 @@ -179,20 +190,21 @@ class MaildirFolder(BaseFolder): os.utime(os.path.join(tmpdir,tmpmessagename), (rtime,rtime)) ui.debug('maildir', 'savemessage: moving from %s to %s' % \ (tmpmessagename, messagename)) - os.link(os.path.join(tmpdir, tmpmessagename), - os.path.join(newdir, messagename)) - os.unlink(os.path.join(tmpdir, tmpmessagename)) + if tmpmessagename != messagename: # then rename it + os.link(os.path.join(tmpdir, tmpmessagename), + os.path.join(tmpdir, messagename)) + os.unlink(os.path.join(tmpdir, tmpmessagename)) try: # fsync the directory (safer semantics in Linux) - fd = os.open(newdir, os.O_RDONLY) + fd = os.open(tmpdir, os.O_RDONLY) os.fsync(fd) os.close(fd) except: pass self.messagelist[uid] = {'uid': uid, 'flags': [], - 'filename': os.path.join(newdir, messagename)} + 'filename': os.path.join(tmpdir, messagename)} self.savemessageflags(uid, flags) ui.debug('maildir', 'savemessage: returning uid %d' % uid) return uid @@ -203,6 +215,7 @@ class MaildirFolder(BaseFolder): def savemessageflags(self, uid, flags): oldfilename = self.messagelist[uid]['filename'] newpath, newname = os.path.split(oldfilename) + tmpdir = os.path.join(self.getfullname(), 'tmp') if 'S' in flags: # If a message has been seen, it goes into the cur # directory. CR debian#152482, [complete.org #4] @@ -225,6 +238,10 @@ class MaildirFolder(BaseFolder): self.messagelist[uid]['flags'] = flags self.messagelist[uid]['filename'] = newfilename + # By now, the message had better not be in tmp/ land! + final_dir, final_name = os.path.split(self.messagelist[uid]['filename']) + assert final_dir != tmpdir + def deletemessage(self, uid): if not uid in self.messagelist: return diff --git a/offlineimap/folder/__init__.py b/offlineimap/folder/__init__.py index bcdb844..425148b 100644 --- a/offlineimap/folder/__init__.py +++ b/offlineimap/folder/__init__.py @@ -1,2 +1,2 @@ -import Base, IMAP, Maildir, LocalStatus +import Base, Gmail, IMAP, Maildir, LocalStatus diff --git a/offlineimap/imaplib.py b/offlineimap/imaplib.py deleted file mode 100644 index d70819e..0000000 --- a/offlineimap/imaplib.py +++ /dev/null @@ -1,1453 +0,0 @@ -"""IMAP4 client. - -Based on RFC 2060. - -Public class: IMAP4 -Public variable: Debug -Public functions: Internaldate2tuple - Internaldate2epoch - Int2AP - ParseFlags - Time2Internaldate -""" - -# Author: Piers Lauder December 1997. -# -# Authentication code contributed by Donn Cave June 1998. -# String method conversion by ESR, February 2001. -# GET/SETACL contributed by Anthony Baxter April 2001. -# IMAP4_SSL contributed by Tino Lange March 2002. -# GET/SETQUOTA contributed by Andreas Zeidler June 2002. -# IMAP4_Tunnel contributed by John Goerzen July 2002 - -__version__ = "2.52" - -import binascii, re, socket, time, random, subprocess, sys, os -from offlineimap.ui import UIBase - -__all__ = ["IMAP4", "Internaldate2tuple", "Internaldate2epoch", - "Int2AP", "ParseFlags", "Time2Internaldate"] - -# Globals - -CRLF = '\r\n' -Debug = 0 -IMAP4_PORT = 143 -IMAP4_SSL_PORT = 993 -AllowedVersions = ('IMAP4REV1', 'IMAP4') # Most recent first - -# Commands - -Commands = { - # name valid states - 'APPEND': ('AUTH', 'SELECTED'), - 'AUTHENTICATE': ('NONAUTH',), - 'CAPABILITY': ('NONAUTH', 'AUTH', 'SELECTED', 'LOGOUT'), - 'CHECK': ('SELECTED',), - 'CLOSE': ('SELECTED',), - 'COPY': ('SELECTED',), - 'CREATE': ('AUTH', 'SELECTED'), - 'DELETE': ('AUTH', 'SELECTED'), - 'EXAMINE': ('AUTH', 'SELECTED'), - 'EXPUNGE': ('SELECTED',), - 'FETCH': ('SELECTED',), - 'GETACL': ('AUTH', 'SELECTED'), - 'GETQUOTA': ('AUTH', 'SELECTED'), - 'GETQUOTAROOT': ('AUTH', 'SELECTED'), - 'LIST': ('AUTH', 'SELECTED'), - 'LOGIN': ('NONAUTH',), - 'LOGOUT': ('NONAUTH', 'AUTH', 'SELECTED', 'LOGOUT'), - 'LSUB': ('AUTH', 'SELECTED'), - 'NAMESPACE': ('AUTH', 'SELECTED'), - 'NOOP': ('NONAUTH', 'AUTH', 'SELECTED', 'LOGOUT'), - 'PARTIAL': ('SELECTED',), # NB: obsolete - 'RENAME': ('AUTH', 'SELECTED'), - 'SEARCH': ('SELECTED',), - 'SELECT': ('AUTH', 'SELECTED'), - 'SETACL': ('AUTH', 'SELECTED'), - 'SETQUOTA': ('AUTH', 'SELECTED'), - 'SORT': ('SELECTED',), - 'STATUS': ('AUTH', 'SELECTED'), - 'STORE': ('SELECTED',), - 'SUBSCRIBE': ('AUTH', 'SELECTED'), - 'UID': ('SELECTED',), - 'UNSUBSCRIBE': ('AUTH', 'SELECTED'), - } - -# Patterns to match server responses - -Continuation = re.compile(r'\+( (?P.*))?') -Flags = re.compile(r'.*FLAGS \((?P[^\)]*)\)') -InternalDate = re.compile(r'.*INTERNALDATE "' - r'(?P[ 0123][0-9])-(?P[A-Z][a-z][a-z])-(?P[0-9][0-9][0-9][0-9])' - r' (?P[0-9][0-9]):(?P[0-9][0-9]):(?P[0-9][0-9])' - r' (?P[-+])(?P[0-9][0-9])(?P[0-9][0-9])' - r'"') -Literal = re.compile(r'.*{(?P\d+)}$') -Response_code = re.compile(r'\[(?P[A-Z-]+)( (?P[^\]]*))?\]') -Untagged_response = re.compile(r'\* (?P[A-Z-]+)( (?P.*))?') -Untagged_status = re.compile(r'\* (?P\d+) (?P[A-Z-]+)( (?P.*))?') - - - -class IMAP4: - - """IMAP4 client class. - - Instantiate with: IMAP4([host[, port]]) - - host - host's name (default: localhost); - port - port number (default: standard IMAP4 port). - - All IMAP4rev1 commands are supported by methods of the same - name (in lower-case). - - All arguments to commands are converted to strings, except for - AUTHENTICATE, and the last argument to APPEND which is passed as - an IMAP4 literal. If necessary (the string contains any - non-printing characters or white-space and isn't enclosed with - either parentheses or double quotes) each string is quoted. - However, the 'password' argument to the LOGIN command is always - quoted. If you want to avoid having an argument string quoted - (eg: the 'flags' argument to STORE) then enclose the string in - parentheses (eg: "(\Deleted)"). - - Each command returns a tuple: (type, [data, ...]) where 'type' - is usually 'OK' or 'NO', and 'data' is either the text from the - tagged response, or untagged results from command. - - Errors raise the exception class .error(""). - IMAP4 server errors raise .abort(""), - which is a sub-class of 'error'. Mailbox status changes - from READ-WRITE to READ-ONLY raise the exception class - .readonly(""), which is a sub-class of 'abort'. - - "error" exceptions imply a program error. - "abort" exceptions imply the connection should be reset, and - the command re-tried. - "readonly" exceptions imply the command should be re-tried. - - Note: to use this module, you must read the RFCs pertaining - to the IMAP4 protocol, as the semantics of the arguments to - each IMAP4 command are left to the invoker, not to mention - the results. - """ - - class error(Exception): pass # Logical errors - debug required - class abort(error): pass # Service errors - close and retry - class readonly(abort): pass # Mailbox status changed to READ-ONLY - - mustquote = re.compile(r"[^\w!#$%&'+,.:;<=>?^`|~-]") - - def __init__(self, host = '', port = IMAP4_PORT): - self.debug = Debug - self.state = 'LOGOUT' - self.literal = None # A literal argument to a command - self.tagged_commands = {} # Tagged commands awaiting response - self.untagged_responses = {} # {typ: [data, ...], ...} - self.continuation_response = '' # Last continuation response - self.is_readonly = None # READ-ONLY desired state - self.tagnum = 0 - - # Open socket to server. - - self.open(host, port) - - # Create unique tag for this session, - # and compile tagged response matcher. - - self.tagpre = Int2AP(random.randint(0, 31999)) - self.tagre = re.compile(r'(?P' - + self.tagpre - + r'\d+) (?P[A-Z]+) (?P.*)') - - # Get server welcome message, - # request and store CAPABILITY response. - - if __debug__: - self._cmd_log_len = 10 - self._cmd_log_idx = 0 - self._cmd_log = {} # Last `_cmd_log_len' interactions - if self.debug >= 1: - self._mesg('imaplib version %s' % __version__) - self._mesg('new IMAP4 connection, tag=%s' % self.tagpre) - - self.welcome = self._get_response() - if 'PREAUTH' in self.untagged_responses: - self.state = 'AUTH' - elif 'OK' in self.untagged_responses: - self.state = 'NONAUTH' - else: - raise self.error(self.welcome) - - cap = 'CAPABILITY' - self._simple_command(cap) - if not cap in self.untagged_responses: - raise self.error('no CAPABILITY response from server') - self.capabilities = tuple(self.untagged_responses[cap][-1].upper().split()) - - if __debug__: - if self.debug >= 3: - self._mesg('CAPABILITIES: %s' % `self.capabilities`) - - for version in AllowedVersions: - if not version in self.capabilities: - continue - self.PROTOCOL_VERSION = version - return - - raise self.error('server not IMAP4 compliant') - - - def __getattr__(self, attr): - # Allow UPPERCASE variants of IMAP4 command methods. - if attr in Commands: - return getattr(self, attr.lower()) - raise AttributeError("Unknown IMAP4 command: '%s'" % attr) - - - - # Overridable methods - - - def open(self, host = '', port = IMAP4_PORT): - """Setup connection to remote server on "host:port" - (default: localhost:standard IMAP4 port). - This connection will be used by the routines: - read, readline, send, shutdown. - """ - self.host = host - self.port = port - res = socket.getaddrinfo(host, port, socket.AF_UNSPEC, - socket.SOCK_STREAM) - self.sock = socket.socket(af, socktype, proto) - - # Try each address returned by getaddrinfo in turn until we - # manage to connect to one. - # Try all the addresses in turn until we connect() - last_error = 0 - for remote in res: - af, socktype, proto, canonname, sa = remote - self.sock = socket.socket(af, socktype, proto) - last_error = self.sock.connect_ex(sa) - print af - if last_error == 0: - break - else: - self.sock.close() - if last_error != 0: - # FIXME - raise socket.error(last_error) - self.file = self.sock.makefile('rb') - - def read(self, size): - """Read 'size' bytes from remote.""" - retval = '' - while len(retval) < size: - retval += self.file.read(size - len(retval)) - return retval - - def readline(self): - """Read line from remote.""" - return self.file.readline() - - - def send(self, data): - """Send data to remote.""" - self.sock.sendall(data) - - - def shutdown(self): - """Close I/O established in "open".""" - self.file.close() - self.sock.close() - - - def socket(self): - """Return socket instance used to connect to IMAP4 server. - - socket = .socket() - """ - return self.sock - - - - # Utility methods - - - def recent(self): - """Return most recent 'RECENT' responses if any exist, - else prompt server for an update using the 'NOOP' command. - - (typ, [data]) = .recent() - - 'data' is None if no new messages, - else list of RECENT responses, most recent last. - """ - name = 'RECENT' - typ, dat = self._untagged_response('OK', [None], name) - if dat[-1]: - return typ, dat - typ, dat = self.noop() # Prod server for response - return self._untagged_response(typ, dat, name) - - - def response(self, code): - """Return data for response 'code' if received, or None. - - Old value for response 'code' is cleared. - - (code, [data]) = .response(code) - """ - return self._untagged_response(code, [None], code.upper()) - - - - # IMAP4 commands - - - def append(self, mailbox, flags, date_time, message): - """Append message to named mailbox. - - (typ, [data]) = .append(mailbox, flags, date_time, message) - - All args except `message' can be None. - """ - name = 'APPEND' - if not mailbox: - mailbox = 'INBOX' - if flags: - if (flags[0],flags[-1]) != ('(',')'): - flags = '(%s)' % flags - else: - flags = None - if date_time: - date_time = Time2Internaldate(date_time) - else: - date_time = None - self.literal = message - return self._simple_command(name, mailbox, flags, date_time) - - - def authenticate(self, mechanism, authobject): - """Authenticate command - requires response processing. - - 'mechanism' specifies which authentication mechanism is to - be used - it must appear in .capabilities in the - form AUTH=. - - 'authobject' must be a callable object: - - data = authobject(response) - - It will be called to process server continuation responses. - It should return data that will be encoded and sent to server. - It should return None if the client abort response '*' should - be sent instead. - """ - mech = mechanism.upper() - cap = 'AUTH=%s' % mech - if not cap in self.capabilities: - raise self.error("Server doesn't allow %s authentication." % mech) - self.literal = _Authenticator(authobject).process - typ, dat = self._simple_command('AUTHENTICATE', mech) - if typ != 'OK': - raise self.error(dat[-1]) - self.state = 'AUTH' - return typ, dat - - - def check(self): - """Checkpoint mailbox on server. - - (typ, [data]) = .check() - """ - return self._simple_command('CHECK') - - - def close(self): - """Close currently selected mailbox. - - Deleted messages are removed from writable mailbox. - This is the recommended command before 'LOGOUT'. - - (typ, [data]) = .close() - """ - try: - typ, dat = self._simple_command('CLOSE') - finally: - self.state = 'AUTH' - return typ, dat - - - def copy(self, message_set, new_mailbox): - """Copy 'message_set' messages onto end of 'new_mailbox'. - - (typ, [data]) = .copy(message_set, new_mailbox) - """ - return self._simple_command('COPY', message_set, new_mailbox) - - - def create(self, mailbox): - """Create new mailbox. - - (typ, [data]) = .create(mailbox) - """ - return self._simple_command('CREATE', mailbox) - - - def delete(self, mailbox): - """Delete old mailbox. - - (typ, [data]) = .delete(mailbox) - """ - return self._simple_command('DELETE', mailbox) - - - def expunge(self): - """Permanently remove deleted items from selected mailbox. - - Generates 'EXPUNGE' response for each deleted message. - - (typ, [data]) = .expunge() - - 'data' is list of 'EXPUNGE'd message numbers in order received. - """ - name = 'EXPUNGE' - typ, dat = self._simple_command(name) - return self._untagged_response(typ, dat, name) - - - def fetch(self, message_set, message_parts): - """Fetch (parts of) messages. - - (typ, [data, ...]) = .fetch(message_set, message_parts) - - 'message_parts' should be a string of selected parts - enclosed in parentheses, eg: "(UID BODY[TEXT])". - - 'data' are tuples of message part envelope and data. - """ - name = 'FETCH' - typ, dat = self._simple_command(name, message_set, message_parts) - return self._untagged_response(typ, dat, name) - - - def getacl(self, mailbox): - """Get the ACLs for a mailbox. - - (typ, [data]) = .getacl(mailbox) - """ - typ, dat = self._simple_command('GETACL', mailbox) - return self._untagged_response(typ, dat, 'ACL') - - - def getquota(self, root): - """Get the quota root's resource usage and limits. - - Part of the IMAP4 QUOTA extension defined in rfc2087. - - (typ, [data]) = .getquota(root) - """ - typ, dat = self._simple_command('GETQUOTA', root) - return self._untagged_response(typ, dat, 'QUOTA') - - - def getquotaroot(self, mailbox): - """Get the list of quota roots for the named mailbox. - - (typ, [[QUOTAROOT responses...], [QUOTA responses]]) = .getquotaroot(mailbox) - """ - typ, dat = self._simple_command('GETQUOTA', root) - typ, quota = self._untagged_response(typ, dat, 'QUOTA') - typ, quotaroot = self._untagged_response(typ, dat, 'QUOTAROOT') - return typ, [quotaroot, quota] - - - def list(self, directory='""', pattern='*'): - """List mailbox names in directory matching pattern. - - (typ, [data]) = .list(directory='""', pattern='*') - - 'data' is list of LIST responses. - """ - name = 'LIST' - typ, dat = self._simple_command(name, directory, pattern) - return self._untagged_response(typ, dat, name) - - - def login(self, user, password): - """Identify client using plaintext password. - - (typ, [data]) = .login(user, password) - - NB: 'password' will be quoted. - """ - #if not 'AUTH=LOGIN' in self.capabilities: - # raise self.error("Server doesn't allow LOGIN authentication." % mech) - typ, dat = self._simple_command('LOGIN', user, self._quote(password)) - if typ != 'OK': - raise self.error(dat[-1]) - self.state = 'AUTH' - return typ, dat - - - def logout(self): - """Shutdown connection to server. - - (typ, [data]) = .logout() - - Returns server 'BYE' response. - """ - self.state = 'LOGOUT' - try: typ, dat = self._simple_command('LOGOUT') - except: typ, dat = 'NO', ['%s: %s' % sys.exc_info()[:2]] - self.shutdown() - if 'BYE' in self.untagged_responses: - return 'BYE', self.untagged_responses['BYE'] - return typ, dat - - - def lsub(self, directory='""', pattern='*'): - """List 'subscribed' mailbox names in directory matching pattern. - - (typ, [data, ...]) = .lsub(directory='""', pattern='*') - - 'data' are tuples of message part envelope and data. - """ - name = 'LSUB' - typ, dat = self._simple_command(name, directory, pattern) - return self._untagged_response(typ, dat, name) - - - def namespace(self): - """ Returns IMAP namespaces ala rfc2342 - - (typ, [data, ...]) = .namespace() - """ - name = 'NAMESPACE' - typ, dat = self._simple_command(name) - return self._untagged_response(typ, dat, name) - - - def noop(self): - """Send NOOP command. - - (typ, data) = .noop() - """ - if __debug__: - if self.debug >= 3: - self._dump_ur(self.untagged_responses) - return self._simple_command('NOOP') - - - def partial(self, message_num, message_part, start, length): - """Fetch truncated part of a message. - - (typ, [data, ...]) = .partial(message_num, message_part, start, length) - - 'data' is tuple of message part envelope and data. - """ - name = 'PARTIAL' - typ, dat = self._simple_command(name, message_num, message_part, start, length) - return self._untagged_response(typ, dat, 'FETCH') - - - def rename(self, oldmailbox, newmailbox): - """Rename old mailbox name to new. - - (typ, data) = .rename(oldmailbox, newmailbox) - """ - return self._simple_command('RENAME', oldmailbox, newmailbox) - - - def search(self, charset, *criteria): - """Search mailbox for matching messages. - - (typ, [data]) = .search(charset, criterium, ...) - - 'data' is space separated list of matching message numbers. - """ - name = 'SEARCH' - if charset: - typ, dat = apply(self._simple_command, (name, 'CHARSET', charset) + criteria) - else: - typ, dat = apply(self._simple_command, (name,) + criteria) - return self._untagged_response(typ, dat, name) - - - def select(self, mailbox='INBOX', readonly=None): - """Select a mailbox. - - Flush all untagged responses. - - (typ, [data]) = .select(mailbox='INBOX', readonly=None) - - 'data' is count of messages in mailbox ('EXISTS' response). - """ - # Mandated responses are ('FLAGS', 'EXISTS', 'RECENT', 'UIDVALIDITY') - self.untagged_responses = {} # Flush old responses. - self.is_readonly = readonly - name = 'SELECT' - typ, dat = self._simple_command(name, mailbox) - if typ != 'OK': - self.state = 'AUTH' # Might have been 'SELECTED' - return typ, dat - self.state = 'SELECTED' - if 'READ-ONLY' in self.untagged_responses \ - and not readonly: - if __debug__: - if self.debug >= 1: - self._dump_ur(self.untagged_responses) - raise self.readonly('%s is not writable' % mailbox) - return typ, self.untagged_responses.get('EXISTS', [None]) - - - def setacl(self, mailbox, who, what): - """Set a mailbox acl. - - (typ, [data]) = .create(mailbox, who, what) - """ - return self._simple_command('SETACL', mailbox, who, what) - - - def setquota(self, root, limits): - """Set the quota root's resource limits. - - (typ, [data]) = .setquota(root, limits) - """ - typ, dat = self._simple_command('SETQUOTA', root, limits) - return self._untagged_response(typ, dat, 'QUOTA') - - - def sort(self, sort_criteria, charset, *search_criteria): - """IMAP4rev1 extension SORT command. - - (typ, [data]) = .sort(sort_criteria, charset, search_criteria, ...) - """ - name = 'SORT' - #if not name in self.capabilities: # Let the server decide! - # raise self.error('unimplemented extension command: %s' % name) - if (sort_criteria[0],sort_criteria[-1]) != ('(',')'): - sort_criteria = '(%s)' % sort_criteria - typ, dat = apply(self._simple_command, (name, sort_criteria, charset) + search_criteria) - return self._untagged_response(typ, dat, name) - - - def status(self, mailbox, names): - """Request named status conditions for mailbox. - - (typ, [data]) = .status(mailbox, names) - """ - name = 'STATUS' - #if self.PROTOCOL_VERSION == 'IMAP4': # Let the server decide! - # raise self.error('%s unimplemented in IMAP4 (obtain IMAP4rev1 server, or re-code)' % name) - typ, dat = self._simple_command(name, mailbox, names) - return self._untagged_response(typ, dat, name) - - - def store(self, message_set, command, flags): - """Alters flag dispositions for messages in mailbox. - - (typ, [data]) = .store(message_set, command, flags) - """ - if (flags[0],flags[-1]) != ('(',')'): - flags = '(%s)' % flags # Avoid quoting the flags - typ, dat = self._simple_command('STORE', message_set, command, flags) - return self._untagged_response(typ, dat, 'FETCH') - - - def subscribe(self, mailbox): - """Subscribe to new mailbox. - - (typ, [data]) = .subscribe(mailbox) - """ - return self._simple_command('SUBSCRIBE', mailbox) - - - def uid(self, command, *args): - """Execute "command arg ..." with messages identified by UID, - rather than message number. - - (typ, [data]) = .uid(command, arg1, arg2, ...) - - Returns response appropriate to 'command'. - """ - command = command.upper() - if not command in Commands: - raise self.error("Unknown IMAP4 UID command: %s" % command) - if self.state not in Commands[command]: - raise self.error('command %s illegal in state %s' - % (command, self.state)) - name = 'UID' - typ, dat = apply(self._simple_command, (name, command) + args) - if command in ('SEARCH', 'SORT'): - name = command - else: - name = 'FETCH' - return self._untagged_response(typ, dat, name) - - - def unsubscribe(self, mailbox): - """Unsubscribe from old mailbox. - - (typ, [data]) = .unsubscribe(mailbox) - """ - return self._simple_command('UNSUBSCRIBE', mailbox) - - - def xatom(self, name, *args): - """Allow simple extension commands - notified by server in CAPABILITY response. - - Assumes command is legal in current state. - - (typ, [data]) = .xatom(name, arg, ...) - - Returns response appropriate to extension command `name'. - """ - name = name.upper() - #if not name in self.capabilities: # Let the server decide! - # raise self.error('unknown extension command: %s' % name) - if not name in Commands: - Commands[name] = (self.state,) - return apply(self._simple_command, (name,) + args) - - - - # Private methods - - - def _append_untagged(self, typ, dat): - - if dat is None: dat = '' - ur = self.untagged_responses - if __debug__: - if self.debug >= 5: - self._mesg('untagged_responses[%s] %s += ["%s"]' % - (typ, len(ur.get(typ,'')), dat)) - if typ in ur: - ur[typ].append(dat) - else: - ur[typ] = [dat] - - - def _check_bye(self): - bye = self.untagged_responses.get('BYE') - if bye: - raise self.abort(bye[-1]) - - - def _command(self, name, *args): - - if self.state not in Commands[name]: - self.literal = None - raise self.error( - 'command %s illegal in state %s' % (name, self.state)) - - for typ in ('OK', 'NO', 'BAD'): - if typ in self.untagged_responses: - del self.untagged_responses[typ] - - if 'READ-ONLY' in self.untagged_responses \ - and not self.is_readonly: - raise self.readonly('mailbox status changed to READ-ONLY') - - tag = self._new_tag() - data = '%s %s' % (tag, name) - for arg in args: - if arg is None: continue - data = '%s %s' % (data, self._checkquote(arg)) - - literal = self.literal - if literal is not None: - self.literal = None - if type(literal) is type(self._command): - literator = literal - else: - literator = None - data = '%s {%s}' % (data, len(literal)) - - if __debug__: - if self.debug >= 4: - self._mesg('> %s' % data) - else: - self._log('> %s' % data) - - try: - self.send('%s%s' % (data, CRLF)) - except (socket.error, OSError), val: - raise self.abort('socket error: %s' % val) - - if literal is None: - return tag - - while 1: - # Wait for continuation response - - while self._get_response(): - if self.tagged_commands[tag]: # BAD/NO? - return tag - - # Send literal - - if literator: - literal = literator(self.continuation_response) - - if __debug__: - if self.debug >= 4: - self._mesg('write literal size %s' % len(literal)) - - try: - self.send(literal) - self.send(CRLF) - except (socket.error, OSError), val: - raise self.abort('socket error: %s' % val) - - if not literator: - break - - return tag - - - def _command_complete(self, name, tag): - self._check_bye() - try: - typ, data = self._get_tagged_response(tag) - except self.abort, val: - raise self.abort('command: %s => %s' % (name, val)) - except self.error, val: - raise self.error('command: %s => %s' % (name, val)) - self._check_bye() - if typ == 'BAD': - raise self.error('%s command error: %s %s' % (name, typ, data)) - return typ, data - - - def _get_response(self): - - # Read response and store. - # - # Returns None for continuation responses, - # otherwise first response line received. - - resp = self._get_line() - - # Command completion response? - - if self._match(self.tagre, resp): - tag = self.mo.group('tag') - if not tag in self.tagged_commands: - raise self.abort('unexpected tagged response: %s' % resp) - - typ = self.mo.group('type') - dat = self.mo.group('data') - self.tagged_commands[tag] = (typ, [dat]) - else: - dat2 = None - - # '*' (untagged) responses? - - if not self._match(Untagged_response, resp): - if self._match(Untagged_status, resp): - dat2 = self.mo.group('data2') - - if self.mo is None: - # Only other possibility is '+' (continuation) response... - - if self._match(Continuation, resp): - self.continuation_response = self.mo.group('data') - return None # NB: indicates continuation - - raise self.abort("unexpected response: '%s'" % resp) - - typ = self.mo.group('type') - dat = self.mo.group('data') - if dat is None: dat = '' # Null untagged response - if dat2: dat = dat + ' ' + dat2 - - # Is there a literal to come? - - while self._match(Literal, dat): - - # Read literal direct from connection. - - size = int(self.mo.group('size')) - if __debug__: - if self.debug >= 4: - self._mesg('read literal size %s' % size) - data = self.read(size) - - # Store response with literal as tuple - - self._append_untagged(typ, (dat, data)) - - # Read trailer - possibly containing another literal - - dat = self._get_line() - - self._append_untagged(typ, dat) - - # Bracketed response information? - - if typ in ('OK', 'NO', 'BAD') and self._match(Response_code, dat): - self._append_untagged(self.mo.group('type'), self.mo.group('data')) - - if __debug__: - if self.debug >= 1 and typ in ('NO', 'BAD', 'BYE'): - self._mesg('%s response: %s' % (typ, dat)) - - return resp - - - def _get_tagged_response(self, tag): - - while 1: - result = self.tagged_commands[tag] - if result is not None: - del self.tagged_commands[tag] - return result - - # Some have reported "unexpected response" exceptions. - # Note that ignoring them here causes loops. - # Instead, send me details of the unexpected response and - # I'll update the code in `_get_response()'. - - try: - self._get_response() - except self.abort, val: - if __debug__: - if self.debug >= 1: - self.print_log() - raise - - - def _get_line(self): - - line = self.readline() - if not line: - raise self.abort('socket error: EOF') - - # Protocol mandates all lines terminated by CRLF - - line = line[:-2] - if __debug__: - if self.debug >= 4: - self._mesg('< %s' % line) - else: - self._log('< %s' % line) - return line - - - def _match(self, cre, s): - - # Run compiled regular expression match method on 's'. - # Save result, return success. - - self.mo = cre.match(s) - if __debug__: - if self.mo is not None and self.debug >= 5: - self._mesg("\tmatched r'%s' => %s" % (cre.pattern, `self.mo.groups()`)) - return self.mo is not None - - - def _new_tag(self): - - tag = '%s%s' % (self.tagpre, self.tagnum) - self.tagnum = self.tagnum + 1 - self.tagged_commands[tag] = None - return tag - - - def _checkquote(self, arg): - - # Must quote command args if non-alphanumeric chars present, - # and not already quoted. - - if type(arg) is not type(''): - return arg - if (arg[0],arg[-1]) in (('(',')'),('"','"')): - return arg - if self.mustquote.search(arg) is None: - return arg - return self._quote(arg) - - - def _quote(self, arg): - - arg = arg.replace('\\', '\\\\') - arg = arg.replace('"', '\\"') - - return '"%s"' % arg - - - def _simple_command(self, name, *args): - - return self._command_complete(name, apply(self._command, (name,) + args)) - - - def _untagged_response(self, typ, dat, name): - - if typ == 'NO': - return typ, dat - if not name in self.untagged_responses: - return typ, [None] - data = self.untagged_responses[name] - if __debug__: - if self.debug >= 5: - self._mesg('untagged_responses[%s] => %s' % (name, data)) - del self.untagged_responses[name] - return typ, data - - - if __debug__: - - def _mesg(self, s, secs=None): - if secs is None: - secs = time.time() - tm = time.strftime('%M:%S', time.localtime(secs)) - UIBase.getglobalui().debug('imap', ' %s.%02d %s' % (tm, (secs*100)%100, s)) - - def _dump_ur(self, dict): - # Dump untagged responses (in `dict'). - l = dict.items() - if not l: return - t = '\n\t\t' - l = map(lambda x:'%s: "%s"' % (x[0], x[1][0] and '" "'.join(x[1]) or ''), l) - self._mesg('untagged responses dump:%s%s' % (t, t.join(l))) - - def _log(self, line): - # Keep log of last `_cmd_log_len' interactions for debugging. - self._cmd_log[self._cmd_log_idx] = (line, time.time()) - self._cmd_log_idx += 1 - if self._cmd_log_idx >= self._cmd_log_len: - self._cmd_log_idx = 0 - - def print_log(self): - self._mesg('last %d IMAP4 interactions:' % len(self._cmd_log)) - i, n = self._cmd_log_idx, self._cmd_log_len - while n: - try: - apply(self._mesg, self._cmd_log[i]) - except: - pass - i += 1 - if i >= self._cmd_log_len: - i = 0 - n -= 1 - -class IMAP4_Tunnel(IMAP4): - """IMAP4 client class over a tunnel - - Instantiate with: IMAP4_Tunnel(tunnelcmd) - - tunnelcmd -- shell command to generate the tunnel. - The result will be in PREAUTH stage.""" - - def __init__(self, tunnelcmd): - IMAP4.__init__(self, tunnelcmd) - - def open(self, host, port): - """The tunnelcmd comes in on host!""" - self.process = subprocess.Popen(host, shell=True, close_fds=True, - stdin=subprocess.PIPE, stdout=subprocess.PIPE) - (self.outfd, self.infd) = (self.process.stdin, self.process.stdout) - - def read(self, size): - retval = '' - while len(retval) < size: - retval += self.infd.read(size - len(retval)) - return retval - - def readline(self): - return self.infd.readline() - - def send(self, data): - self.outfd.write(data) - - def shutdown(self): - self.infd.close() - self.outfd.close() - self.process.wait() - - -class sslwrapper: - def __init__(self, sslsock): - self.sslsock = sslsock - self.readbuf = '' - - def write(self, s): - return self.sslsock.write(s) - - def _read(self, n): - return self.sslsock.read(n) - - def read(self, n): - if len(self.readbuf): - # Return the stuff in readbuf, even if less than n. - # It might contain the rest of the line, and if we try to - # read more, might block waiting for data that is not - # coming to arrive. - bytesfrombuf = min(n, len(self.readbuf)) - retval = self.readbuf[:bytesfrombuf] - self.readbuf = self.readbuf[bytesfrombuf:] - return retval - retval = self._read(n) - if len(retval) > n: - self.readbuf = retval[n:] - return retval[:n] - return retval - - def readline(self): - retval = '' - while 1: - linebuf = self.read(1024) - nlindex = linebuf.find("\n") - if nlindex != -1: - retval += linebuf[:nlindex + 1] - self.readbuf = linebuf[nlindex + 1:] + self.readbuf - return retval - else: - retval += linebuf - - -class IMAP4_SSL(IMAP4): - - """IMAP4 client class over SSL connection - - Instantiate with: IMAP4_SSL([host[, port[, keyfile[, certfile]]]]) - - host - host's name (default: localhost); - port - port number (default: standard IMAP4 SSL port). - keyfile - PEM formatted file that contains your private key (default: None); - certfile - PEM formatted certificate chain file (default: None); - - for more documentation see the docstring of the parent class IMAP4. - """ - - - def __init__(self, host = '', port = IMAP4_SSL_PORT, keyfile = None, certfile = None): - self.keyfile = keyfile - self.certfile = certfile - IMAP4.__init__(self, host, port) - - - def open(self, host = '', port = IMAP4_SSL_PORT): - """Setup connection to remote server on "host:port". - (default: localhost:standard IMAP4 SSL port). - This connection will be used by the routines: - read, readline, send, shutdown. - """ - self.host = host - self.port = port - #This connects to the first ip found ipv4/ipv6 - #Added by Adriaan Peeters based on a socket - #example from the python documentation: - #http://www.python.org/doc/lib/socket-example.html - res = socket.getaddrinfo(host, port, socket.AF_UNSPEC, - socket.SOCK_STREAM) - # Try all the addresses in turn until we connect() - last_error = 0 - for remote in res: - af, socktype, proto, canonname, sa = remote - self.sock = socket.socket(af, socktype, proto) - last_error = self.sock.connect_ex(sa) - print af - if last_error == 0: - break - else: - self.sock.close() - if last_error != 0: - # FIXME - raise socket.error(last_error) - if sys.version_info[0] <= 2 and sys.version_info[1] <= 2: - self.sslobj = socket.ssl(self.sock, self.keyfile, self.certfile) - else: - self.sslobj = socket.ssl(self.sock._sock, self.keyfile, self.certfile) - self.sslobj = sslwrapper(self.sslobj) - - - def read(self, size): - """Read 'size' bytes from remote.""" - retval = '' - while len(retval) < size: - retval += self.sslobj.read(size - len(retval)) - return retval - - - def readline(self): - """Read line from remote.""" - return self.sslobj.readline() - - def send(self, data): - """Send data to remote.""" - byteswritten = 0 - bytestowrite = len(data) - while byteswritten < bytestowrite: - byteswritten += self.sslobj.write(data[byteswritten:]) - - - def shutdown(self): - """Close I/O established in "open".""" - self.sock.close() - - - def socket(self): - """Return socket instance used to connect to IMAP4 server. - - socket = .socket() - """ - return self.sock - - - def ssl(self): - """Return SSLObject instance used to communicate with the IMAP4 server. - - ssl = .socket.ssl() - """ - return self.sslobj - - - -class _Authenticator: - - """Private class to provide en/decoding - for base64-based authentication conversation. - """ - - def __init__(self, mechinst): - self.mech = mechinst # Callable object to provide/process data - - def process(self, data): - ret = self.mech(self.decode(data)) - if ret is None: - return '*' # Abort conversation - return self.encode(ret) - - def encode(self, inp): - # - # Invoke binascii.b2a_base64 iteratively with - # short even length buffers, strip the trailing - # line feed from the result and append. "Even" - # means a number that factors to both 6 and 8, - # so when it gets to the end of the 8-bit input - # there's no partial 6-bit output. - # - oup = '' - while inp: - if len(inp) > 48: - t = inp[:48] - inp = inp[48:] - else: - t = inp - inp = '' - e = binascii.b2a_base64(t) - if e: - oup = oup + e[:-1] - return oup - - def decode(self, inp): - if not inp: - return '' - return binascii.a2b_base64(inp) - - - -Mon2num = {'Jan': 1, 'Feb': 2, 'Mar': 3, 'Apr': 4, 'May': 5, 'Jun': 6, - 'Jul': 7, 'Aug': 8, 'Sep': 9, 'Oct': 10, 'Nov': 11, 'Dec': 12} - -def Internaldate2epoch(resp): - """Convert IMAP4 INTERNALDATE to UT. - - Returns seconds since the epoch. - """ - - mo = InternalDate.match(resp) - if not mo: - return None - - mon = Mon2num[mo.group('mon')] - zonen = mo.group('zonen') - - day = int(mo.group('day')) - year = int(mo.group('year')) - hour = int(mo.group('hour')) - min = int(mo.group('min')) - sec = int(mo.group('sec')) - zoneh = int(mo.group('zoneh')) - zonem = int(mo.group('zonem')) - - # INTERNALDATE timezone must be subtracted to get UT - - zone = (zoneh*60 + zonem)*60 - if zonen == '-': - zone = -zone - - tt = (year, mon, day, hour, min, sec, -1, -1, -1) - - return time.mktime(tt) - - -def Internaldate2tuple(resp): - """Convert IMAP4 INTERNALDATE to UT. - - Returns Python time module tuple. - """ - - utc = Internaldate2epoch(resp) - - # Following is necessary because the time module has no 'mkgmtime'. - # 'mktime' assumes arg in local timezone, so adds timezone/altzone. - - lt = time.localtime(utc) - if time.daylight and lt[-1]: - zone = zone + time.altzone - else: - zone = zone + time.timezone - - return time.localtime(utc - zone) - - - -def Int2AP(num): - - """Convert integer to A-P string representation.""" - - val = ''; AP = 'ABCDEFGHIJKLMNOP' - num = int(abs(num)) - while num: - num, mod = divmod(num, 16) - val = AP[mod] + val - return val - - - -def ParseFlags(resp): - - """Convert IMAP4 flags response to python tuple.""" - - mo = Flags.match(resp) - if not mo: - return () - - return tuple(mo.group('flags').split()) - - -def Time2Internaldate(date_time): - - """Convert 'date_time' to IMAP4 INTERNALDATE representation. - - Return string in form: '"DD-Mmm-YYYY HH:MM:SS +HHMM"' - """ - - if isinstance(date_time, (int, float)): - tt = time.localtime(date_time) - elif isinstance(date_time, (tuple, time.struct_time)): - tt = date_time - elif isinstance(date_time, str) and (date_time[0],date_time[-1]) == ('"','"'): - return date_time # Assume in correct format - else: - raise ValueError("date_time not of a known type") - - dt = time.strftime("%d-%b-%Y %H:%M:%S", tt) - if dt[0] == '0': - dt = ' ' + dt[1:] - if time.daylight and tt[-1]: - zone = -time.altzone - else: - zone = -time.timezone - return '"' + dt + " %+03d%02d" % divmod(zone/60, 60) + '"' - - - -if __name__ == '__main__': - - import getopt, getpass - - try: - optlist, args = getopt.getopt(sys.argv[1:], 'd:') - except getopt.error, val: - pass - - for opt,val in optlist: - if opt == '-d': - Debug = int(val) - - if not args: args = ('',) - - host = args[0] - - USER = getpass.getuser() - PASSWD = getpass.getpass("IMAP password for %s on %s: " % (USER, host or "localhost")) - - test_mesg = 'From: %(user)s@localhost%(lf)sSubject: IMAP4 test%(lf)s%(lf)sdata...%(lf)s' % {'user':USER, 'lf':CRLF} - test_seq1 = ( - ('login', (USER, PASSWD)), - ('create', ('/tmp/xxx 1',)), - ('rename', ('/tmp/xxx 1', '/tmp/yyy')), - ('CREATE', ('/tmp/yyz 2',)), - ('append', ('/tmp/yyz 2', None, None, test_mesg)), - ('list', ('/tmp', 'yy*')), - ('select', ('/tmp/yyz 2',)), - ('search', (None, 'SUBJECT', 'test')), - ('fetch', ('1', '(FLAGS INTERNALDATE RFC822)')), - ('store', ('1', 'FLAGS', '(\Deleted)')), - ('namespace', ()), - ('expunge', ()), - ('recent', ()), - ('close', ()), - ) - - test_seq2 = ( - ('select', ()), - ('response',('UIDVALIDITY',)), - ('uid', ('SEARCH', 'ALL')), - ('response', ('EXISTS',)), - ('append', (None, None, None, test_mesg)), - ('recent', ()), - ('logout', ()), - ) - - def run(cmd, args): - M._mesg('%s %s' % (cmd, args)) - typ, dat = apply(getattr(M, cmd), args) - M._mesg('%s => %s %s' % (cmd, typ, dat)) - return dat - - try: - M = IMAP4(host) - M._mesg('PROTOCOL_VERSION = %s' % M.PROTOCOL_VERSION) - M._mesg('CAPABILITIES = %s' % `M.capabilities`) - - for cmd,args in test_seq1: - run(cmd, args) - - for ml in run('list', ('/tmp/', 'yy%')): - mo = re.match(r'.*"([^"]+)"$', ml) - if mo: path = mo.group(1) - else: path = ml.split()[-1] - run('delete', (path,)) - - for cmd,args in test_seq2: - dat = run(cmd, args) - - if (cmd,args) != ('uid', ('SEARCH', 'ALL')): - continue - - uid = dat[-1].split() - if not uid: continue - run('uid', ('FETCH', '%s' % uid[-1], - '(FLAGS INTERNALDATE RFC822.SIZE RFC822.HEADER RFC822.TEXT)')) - - print '\nAll tests OK.' - - except: - print '\nTests failed.' - - if not Debug: - print ''' -If you would like to see debugging output, -try: %s -d5 -''' % sys.argv[0] - - raise diff --git a/offlineimap/imaplibutil.py b/offlineimap/imaplibutil.py new file mode 100644 index 0000000..23ba654 --- /dev/null +++ b/offlineimap/imaplibutil.py @@ -0,0 +1,206 @@ +# imaplib utilities +# Copyright (C) 2002-2007 John Goerzen +# +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + +import re, string, types, binascii, socket, time, random, subprocess, sys, os +from offlineimap.ui import UIBase +from imaplib import * + +# Import the symbols we need that aren't exported by default +from imaplib import IMAP4_PORT, IMAP4_SSL_PORT, InternalDate, Mon2num + + +class IMAP4_Tunnel(IMAP4): + """IMAP4 client class over a tunnel + + Instantiate with: IMAP4_Tunnel(tunnelcmd) + + tunnelcmd -- shell command to generate the tunnel. + The result will be in PREAUTH stage.""" + + def __init__(self, tunnelcmd): + IMAP4.__init__(self, tunnelcmd) + + def open(self, host, port): + """The tunnelcmd comes in on host!""" + self.process = subprocess.Popen(host, shell=True, close_fds=True, + stdin=subprocess.PIPE, stdout=subprocess.PIPE) + (self.outfd, self.infd) = (self.process.stdin, self.process.stdout) + + def read(self, size): + retval = '' + while len(retval) < size: + retval += self.infd.read(size - len(retval)) + return retval + + def readline(self): + return self.infd.readline() + + def send(self, data): + self.outfd.write(data) + + def shutdown(self): + self.infd.close() + self.outfd.close() + self.process.wait() + +class sslwrapper: + def __init__(self, sslsock): + self.sslsock = sslsock + self.readbuf = '' + + def write(self, s): + return self.sslsock.write(s) + + def _read(self, n): + return self.sslsock.read(n) + + def read(self, n): + if len(self.readbuf): + # Return the stuff in readbuf, even if less than n. + # It might contain the rest of the line, and if we try to + # read more, might block waiting for data that is not + # coming to arrive. + bytesfrombuf = min(n, len(self.readbuf)) + retval = self.readbuf[:bytesfrombuf] + self.readbuf = self.readbuf[bytesfrombuf:] + return retval + retval = self._read(n) + if len(retval) > n: + self.readbuf = retval[n:] + return retval[:n] + return retval + + def readline(self): + retval = '' + while 1: + linebuf = self.read(1024) + nlindex = linebuf.find("\n") + if nlindex != -1: + retval += linebuf[:nlindex + 1] + self.readbuf = linebuf[nlindex + 1:] + self.readbuf + return retval + else: + retval += linebuf + +def new_mesg(self, s, secs=None): + if secs is None: + secs = time.time() + tm = time.strftime('%M:%S', time.localtime(secs)) + UIBase.getglobalui().debug('imap', ' %s.%02d %s' % (tm, (secs*100)%100, s)) + +class WrappedIMAP4_SSL(IMAP4_SSL): + def open(self, host = '', port = IMAP4_SSL_PORT): + IMAP4_SSL.open(self, host, port) + self.sslobj = sslwrapper(self.sslobj) + + def readline(self): + return self.sslobj.readline() + +def new_open(self, host = '', port = IMAP4_PORT): + """Setup connection to remote server on "host:port" + (default: localhost:standard IMAP4 port). + This connection will be used by the routines: + read, readline, send, shutdown. + """ + self.host = host + self.port = port + res = socket.getaddrinfo(host, port, socket.AF_UNSPEC, + socket.SOCK_STREAM) + + # Try each address returned by getaddrinfo in turn until we + # manage to connect to one. + # Try all the addresses in turn until we connect() + last_error = 0 + for remote in res: + af, socktype, proto, canonname, sa = remote + self.sock = socket.socket(af, socktype, proto) + last_error = self.sock.connect_ex(sa) + if last_error == 0: + break + else: + self.sock.close() + if last_error != 0: + # FIXME + raise socket.error(last_error) + self.file = self.sock.makefile('rb') + +def new_open_ssl(self, host = '', port = IMAP4_SSL_PORT): + """Setup connection to remote server on "host:port". + (default: localhost:standard IMAP4 SSL port). + This connection will be used by the routines: + read, readline, send, shutdown. + """ + self.host = host + self.port = port + #This connects to the first ip found ipv4/ipv6 + #Added by Adriaan Peeters based on a socket + #example from the python documentation: + #http://www.python.org/doc/lib/socket-example.html + res = socket.getaddrinfo(host, port, socket.AF_UNSPEC, + socket.SOCK_STREAM) + # Try all the addresses in turn until we connect() + last_error = 0 + for remote in res: + af, socktype, proto, canonname, sa = remote + self.sock = socket.socket(af, socktype, proto) + last_error = self.sock.connect_ex(sa) + if last_error == 0: + break + else: + self.sock.close() + if last_error != 0: + # FIXME + raise socket.error(last_error) + if sys.version_info[0] <= 2 and sys.version_info[1] <= 2: + self.sslobj = socket.ssl(self.sock, self.keyfile, self.certfile) + else: + self.sslobj = socket.ssl(self.sock._sock, self.keyfile, self.certfile) + self.sslobj = sslwrapper(self.sslobj) + +mustquote = re.compile(r"[^\w!#$%&'+,.:;<=>?^`|~-]") + +def Internaldate2epoch(resp): + """Convert IMAP4 INTERNALDATE to UT. + + Returns seconds since the epoch. + """ + + mo = InternalDate.match(resp) + if not mo: + return None + + mon = Mon2num[mo.group('mon')] + zonen = mo.group('zonen') + + day = int(mo.group('day')) + year = int(mo.group('year')) + hour = int(mo.group('hour')) + min = int(mo.group('min')) + sec = int(mo.group('sec')) + zoneh = int(mo.group('zoneh')) + zonem = int(mo.group('zonem')) + + # INTERNALDATE timezone must be subtracted to get UT + + zone = (zoneh*60 + zonem)*60 + if zonen == '-': + zone = -zone + + tt = (year, mon, day, hour, min, sec, -1, -1, -1) + + return time.mktime(tt) diff --git a/offlineimap/imapserver.py b/offlineimap/imapserver.py index 9ebdb2e..6ca1fbc 100644 --- a/offlineimap/imapserver.py +++ b/offlineimap/imapserver.py @@ -1,5 +1,5 @@ # IMAP server support -# Copyright (C) 2002, 2003 John Goerzen +# Copyright (C) 2002 - 2007 John Goerzen # # # This program is free software; you can redistribute it and/or modify @@ -16,7 +16,8 @@ # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA -from offlineimap import imaplib, imaputil, threadutil +import imaplib +from offlineimap import imaplibutil, imaputil, threadutil from offlineimap.ui import UIBase from threading import * import thread, hmac, os @@ -31,8 +32,8 @@ class UsefulIMAPMixIn: return None def select(self, mailbox='INBOX', readonly=None, force = 0): - if (not force) and self.getselectedfolder() == mailbox: - self.is_readonly = readonly + if (not force) and self.getselectedfolder() == mailbox \ + and self.is_readonly == readonly: # No change; return. return result = self.__class__.__bases__[1].select(self, mailbox, readonly) @@ -43,9 +44,18 @@ class UsefulIMAPMixIn: else: self.selectedfolder = None -class UsefulIMAP4(UsefulIMAPMixIn, imaplib.IMAP4): pass -class UsefulIMAP4_SSL(UsefulIMAPMixIn, imaplib.IMAP4_SSL): pass -class UsefulIMAP4_Tunnel(UsefulIMAPMixIn, imaplib.IMAP4_Tunnel): pass + def _mesg(self, s, secs=None): + imaplibutil.new_mesg(self, s, secs) + +class UsefulIMAP4(UsefulIMAPMixIn, imaplib.IMAP4): + def open(self, host = '', port = imaplib.IMAP4_PORT): + imaplibutil.new_open(self, host, port) + +class UsefulIMAP4_SSL(UsefulIMAPMixIn, imaplibutil.WrappedIMAP4_SSL): + def open(self, host = '', port = imaplib.IMAP4_SSL_PORT): + imaplibutil.new_open_ssl(self, host, port) + +class UsefulIMAP4_Tunnel(UsefulIMAPMixIn, imaplibutil.IMAP4_Tunnel): pass class IMAPServer: def __init__(self, config, reposname, @@ -57,6 +67,7 @@ class IMAPServer: self.username = username self.password = password self.passworderror = None + self.goodpassword = None self.hostname = hostname self.tunnel = tunnel self.port = port @@ -77,6 +88,9 @@ class IMAPServer: self.reference = reference def getpassword(self): + if self.goodpassword != None: + return self.goodpassword + if self.password != None and self.passworderror == None: return self.password @@ -99,6 +113,7 @@ class IMAPServer: def releaseconnection(self, connection): + """Releases a connection, returning it to the pool.""" self.connectionlock.acquire() self.assignedconnections.remove(connection) self.availableconnections.append(connection) @@ -167,6 +182,8 @@ class IMAPServer: UIBase.getglobalui().connecting(self.hostname, self.port) imapobj = UsefulIMAP4(self.hostname, self.port) + imapobj.mustquote = imaplibutil.mustquote + if not self.tunnel: try: if 'AUTH=CRAM-MD5' in imapobj.capabilities: @@ -180,6 +197,7 @@ class IMAPServer: self.plainauth(imapobj) # Would bail by here if there was a failure. success = 1 + self.goodpassword = self.password except imapobj.error, val: self.passworderror = str(val) self.password = None diff --git a/offlineimap/init.py b/offlineimap/init.py index 736f785..ca09919 100644 --- a/offlineimap/init.py +++ b/offlineimap/init.py @@ -1,5 +1,5 @@ # OfflineIMAP initialization code -# Copyright (C) 2002, 2003 John Goerzen +# Copyright (C) 2002-2007 John Goerzen # # # This program is free software; you can redistribute it and/or modify @@ -16,14 +16,15 @@ # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA -from offlineimap import imaplib, imapserver, repository, folder, mbnames, threadutil, version, syncmaster, accounts +import imaplib +from offlineimap import imapserver, repository, folder, mbnames, threadutil, version, syncmaster, accounts from offlineimap.localeval import LocalEval from offlineimap.threadutil import InstanceLimitedThread, ExitNotifyThread from offlineimap.ui import UIBase import re, os, os.path, offlineimap, sys from offlineimap.CustomConfig import CustomConfigParser from threading import * -import threading +import threading, socket from getopt import getopt try: @@ -49,14 +50,14 @@ def startup(versionno): assert versionno == version.versionstr, "Revision of main program (%s) does not match that of library (%s). Please double-check your PYTHONPATH and installation locations." % (versionno, version.versionstr) options = {} if '--help' in sys.argv[1:]: - sys.stdout.write(version.cmdhelp + "\n") + sys.stdout.write(version.getcmdhelp() + "\n") sys.exit(0) - for optlist in getopt(sys.argv[1:], 'P:1oa:c:d:l:u:h')[0]: + for optlist in getopt(sys.argv[1:], 'P:1oqa:c:d:l:u:hk:f:')[0]: options[optlist[0]] = optlist[1] if options.has_key('-h'): - sys.stdout.write(version.cmdhelp) + sys.stdout.write(version.getcmdhelp()) sys.stdout.write("\n") sys.exit(0) configfilename = os.path.expanduser("~/.offlineimaprc") @@ -78,6 +79,17 @@ def startup(versionno): config.read(configfilename) + # override config values with option '-k' + for option in options.keys(): + if option == '-k': + (key, value) = options['-k'].split('=', 1) + if ':' in key: + (secname, key) = key.split(':', 1) + section = secname.replace("_", " ") + else: + section = "general" + config.set(section, key, value) + ui = offlineimap.ui.detector.findUI(config, options.get('-u')) UIBase.setglobalui(ui) @@ -99,12 +111,41 @@ def startup(versionno): for section in accounts.getaccountlist(config): config.remove_option('Account ' + section, "autorefresh") + if options.has_key('-q'): + for section in accounts.getaccountlist(config): + config.set('Account ' + section, "quick", '-1') + + if options.has_key('-f'): + foldernames = options['-f'].replace(" ", "").split(",") + folderfilter = "lambda f: f in %s" % foldernames + folderincludes = "[]" + for accountname in accounts.getaccountlist(config): + account_section = 'Account ' + accountname + remote_repo_section = 'Repository ' + \ + config.get(account_section, 'remoterepository') + local_repo_section = 'Repository ' + \ + config.get(account_section, 'localrepository') + for section in [remote_repo_section, local_repo_section]: + config.set(section, "folderfilter", folderfilter) + config.set(section, "folderincludes", folderincludes) + lock(config, ui) + try: + pidfd = open(config.getmetadatadir() + "/pid", "w") + pidfd.write(os.getpid()) + pidfd.close() + except: + pass + try: if options.has_key('-l'): sys.stderr = ui.logfile + socktimeout = config.getdefaultint("general", "socktimeout", 0) + if socktimeout > 0: + socket.setdefaulttimeout(socktimeout) + activeaccounts = config.get("general", "accounts") if options.has_key('-a'): activeaccounts = options['-a'] diff --git a/offlineimap/repository/Base.py b/offlineimap/repository/Base.py index 94bce09..0486dfc 100644 --- a/offlineimap/repository/Base.py +++ b/offlineimap/repository/Base.py @@ -1,5 +1,5 @@ # Base repository support -# Copyright (C) 2002, 2003, 2006 John Goerzen +# Copyright (C) 2002-2007 John Goerzen # # # This program is free software; you can redistribute it and/or modify @@ -20,11 +20,13 @@ from offlineimap import CustomConfig import os.path def LoadRepository(name, account, reqtype): + from offlineimap.repository.Gmail import GmailRepository from offlineimap.repository.IMAP import IMAPRepository, MappedIMAPRepository from offlineimap.repository.Maildir import MaildirRepository if reqtype == 'remote': # For now, we don't support Maildirs on the remote side. - typemap = {'IMAP': IMAPRepository} + typemap = {'IMAP': IMAPRepository, + 'Gmail': GmailRepository} elif reqtype == 'local': typemap = {'IMAP': MappedIMAPRepository, 'Maildir': MaildirRepository} @@ -63,6 +65,15 @@ class BaseRepository(CustomConfig.ConfigHelperMixin): return self.restore_folder_atimes() + def connect(self): + """Establish a connection to the remote, if necessary. This exists + so that IMAP connections can all be established up front, gathering + passwords as needed. It was added in order to support the + error recovery -- we need to connect first outside of the error + trap in order to validate the password, and that's the point of + this function.""" + pass + def holdordropconnections(self): pass @@ -97,6 +108,11 @@ class BaseRepository(CustomConfig.ConfigHelperMixin): """Returns a list of ALL folders on this server.""" return [] + def forgetfolders(self): + """Forgets the cached list of folders, if any. Useful to run + after a sync run.""" + pass + def getsep(self): raise NotImplementedError @@ -109,9 +125,12 @@ class BaseRepository(CustomConfig.ConfigHelperMixin): def getfolder(self, foldername): raise NotImplementedError - def syncfoldersto(self, dest): + def syncfoldersto(self, dest, copyfolders): """Syncs the folders in this repository to those in dest. - It does NOT sync the contents of those folders.""" + It does NOT sync the contents of those folders. + + For every time dest.makefolder() is called, also call makefolder() + on each folder in copyfolders.""" src = self srcfolders = src.getfolders() destfolders = dest.getfolders() @@ -134,6 +153,8 @@ class BaseRepository(CustomConfig.ConfigHelperMixin): for key in srchash.keys(): if not key in desthash: dest.makefolder(key) + for copyfolder in copyfolders: + copyfolder.makefolder(key) # # Find deleted folders. diff --git a/offlineimap/repository/Gmail.py b/offlineimap/repository/Gmail.py new file mode 100644 index 0000000..f26e7b9 --- /dev/null +++ b/offlineimap/repository/Gmail.py @@ -0,0 +1,68 @@ +# Gmail IMAP repository support +# Copyright (C) 2008 Riccardo Murri +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + +from IMAP import IMAPRepository +from offlineimap import folder, imaputil +from offlineimap.imapserver import IMAPServer + +class GmailRepository(IMAPRepository): + """Gmail IMAP repository. + + Uses hard-coded host name and port, see: + http://mail.google.com/support/bin/answer.py?answer=78799&topic=12814 + """ + + #: Gmail IMAP server hostname + HOSTNAME = "imap.gmail.com" + + #: Gmail IMAP server port + PORT = 993 + + def __init__(self, reposname, account): + """Initialize a GmailRepository object.""" + account.getconfig().set('Repository ' + reposname, + 'remotehost', GmailRepository.HOSTNAME) + account.getconfig().set('Repository ' + reposname, + 'remoteport', GmailRepository.PORT) + account.getconfig().set('Repository ' + reposname, + 'ssl', 'yes') + IMAPRepository.__init__(self, reposname, account) + + def gethost(self): + return GmailRepository.HOSTNAME + + def getport(self): + return GmailRepository.PORT + + def getssl(self): + return 1 + + def getpreauthtunnel(self): + return None + + def getfolder(self, foldername): + return self.getfoldertype()(self.imapserver, foldername, + self.nametrans(foldername), + self.accountname, self) + + def getfoldertype(self): + return folder.Gmail.GmailFolder + + def getrealdelete(self, foldername): + # XXX: `foldername` is currently ignored - the `realdelete` + # setting is repository-wide + return self.getconfboolean('realdelete', 0) diff --git a/offlineimap/repository/IMAP.py b/offlineimap/repository/IMAP.py index bed0358..8587040 100644 --- a/offlineimap/repository/IMAP.py +++ b/offlineimap/repository/IMAP.py @@ -109,6 +109,7 @@ class IMAPRepository(BaseRepository): user = self.getconf('remoteuser') if user != None: return user + try: netrcentry = netrc.netrc().authentificator(self.gethost()) except IOError, inst: @@ -154,6 +155,7 @@ class IMAPRepository(BaseRepository): password = fd.readline().strip() fd.close() return password + try: netrcentry = netrc.netrc().authenticators(self.gethost()) except IOError, inst: @@ -174,6 +176,13 @@ class IMAPRepository(BaseRepository): def getfoldertype(self): return folder.IMAP.IMAPFolder + def connect(self): + imapobj = self.imapserver.acquireconnection() + self.imapserver.releaseconnection(imapobj) + + def forgetfolders(self): + self.folders = None + def getfolders(self): if self.folders != None: return self.folders diff --git a/offlineimap/repository/LocalStatus.py b/offlineimap/repository/LocalStatus.py index c30e1f1..0a13706 100644 --- a/offlineimap/repository/LocalStatus.py +++ b/offlineimap/repository/LocalStatus.py @@ -37,8 +37,8 @@ class LocalStatusRepository(BaseRepository): return os.path.join(self.directory, foldername) def makefolder(self, foldername): - # "touch" the file. - file = open(self.getfolderfilename(foldername), "ab") + # "touch" the file, truncating it. + file = open(self.getfolderfilename(foldername), "wb") file.close() # Invalidate the cache. self.folders = None diff --git a/offlineimap/repository/__init__.py b/offlineimap/repository/__init__.py index feac780..be5c29e 100644 --- a/offlineimap/repository/__init__.py +++ b/offlineimap/repository/__init__.py @@ -1 +1 @@ -__all__ = ['IMAP', 'Base', 'Maildir', 'LocalStatus'] +__all__ = ['Gmail', 'IMAP', 'Base', 'Maildir', 'LocalStatus'] diff --git a/offlineimap/syncmaster.py b/offlineimap/syncmaster.py index d10c5c3..64f2380 100644 --- a/offlineimap/syncmaster.py +++ b/offlineimap/syncmaster.py @@ -1,5 +1,5 @@ # OfflineIMAP synchronization master code -# Copyright (C) 2002 John Goerzen +# Copyright (C) 2002-2007 John Goerzen # # # This program is free software; you can redistribute it and/or modify @@ -16,7 +16,8 @@ # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA -from offlineimap import imaplib, imapserver, repository, folder, mbnames, threadutil, version +import imaplib +from offlineimap import imapserver, repository, folder, mbnames, threadutil, version from offlineimap.threadutil import InstanceLimitedThread, ExitNotifyThread import offlineimap.accounts from offlineimap.accounts import SyncableAccount diff --git a/offlineimap/ui/Blinkenlights.py b/offlineimap/ui/Blinkenlights.py index 717e81b..6982351 100644 --- a/offlineimap/ui/Blinkenlights.py +++ b/offlineimap/ui/Blinkenlights.py @@ -42,6 +42,10 @@ class BlinkenBase: s.gettf().setcolor('cyan') s.__class__.__bases__[-1].syncingfolder(s, srcrepos, srcfolder, destrepos, destfolder) + def skippingfolder(s, folder): + s.gettf().setcolor('cyan') + s.__class__.__bases__[-1].skippingfolder(s, folder) + def loadmessagelist(s, repos, folder): s.gettf().setcolor('green') s._msg("Scanning folder [%s/%s]" % (s.getnicename(repos), @@ -71,6 +75,13 @@ class BlinkenBase: s.gettf().setcolor('pink') s.__class__.__bases__[-1].deletingflags(s, uidlist, flags, destlist) + def warn(s, msg, minor = 0): + if minor: + s.gettf().setcolor('pink') + else: + s.gettf().setcolor('red') + s.__class__.__bases__[-1].warn(s, msg, minor) + def init_banner(s): s.availablethreadframes = {} s.threadframes = {} diff --git a/offlineimap/ui/Curses.py b/offlineimap/ui/Curses.py index 3ebce9d..4cf9066 100644 --- a/offlineimap/ui/Curses.py +++ b/offlineimap/ui/Curses.py @@ -126,10 +126,11 @@ class CursesUtil: self.start() class CursesAccountFrame: - def __init__(s, master, accountname): + def __init__(s, master, accountname, ui): s.c = master s.children = [] s.accountname = accountname + s.ui = ui def drawleadstr(s, secs = None): if secs == None: @@ -150,7 +151,7 @@ class CursesAccountFrame: s.location += 1 def getnewthreadframe(s): - tf = CursesThreadFrame(s.c, s.window, 0, s.location) + tf = CursesThreadFrame(s.c, s.ui, s.window, 0, s.location) s.location += 1 s.children.append(tf) return tf @@ -180,9 +181,10 @@ class CursesAccountFrame: s.sleeping_abort = 1 class CursesThreadFrame: - def __init__(s, master, window, y, x): + def __init__(s, master, ui, window, y, x): """master should be a CursesUtil object.""" s.c = master + s.ui = ui s.window = window s.x = x s.y = y @@ -212,7 +214,7 @@ class CursesThreadFrame: if self.getcolor() == 'black': self.window.addstr(self.y, self.x, ' ', self.color) else: - self.window.addstr(self.y, self.x, '.', self.color) + self.window.addstr(self.y, self.x, self.ui.config.getdefault("ui.Curses.Blinkenlights", "statuschar", '.'), self.color) self.c.stdscr.move(self.c.height - 1, self.c.width - 1) self.window.refresh() self.c.locked(lockedstuff) @@ -476,7 +478,7 @@ class Blinkenlights(BlinkenBase, UIBase): return s.af[accountname] # New one. - s.af[accountname] = CursesAccountFrame(s.c, accountname) + s.af[accountname] = CursesAccountFrame(s.c, accountname, s) s.c.lock() try: s.c.reset() @@ -509,6 +511,8 @@ class Blinkenlights(BlinkenBase, UIBase): return if color: s.gettf().setcolor(color) + elif s.gettf().getcolor() == 'black': + s.gettf().setcolor('gray') s._addline(msg, s.gettf().getcolorpair()) s.logwindow.refresh() finally: diff --git a/offlineimap/ui/Machine.py b/offlineimap/ui/Machine.py new file mode 100644 index 0000000..d02bbbc --- /dev/null +++ b/offlineimap/ui/Machine.py @@ -0,0 +1,177 @@ +# Copyright (C) 2007 John Goerzen +# +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + +import offlineimap.version +import urllib, sys, re, time, traceback, threading, thread +from UIBase import UIBase +from threading import * + +protocol = '6.0.0' + +class MachineUI(UIBase): + def __init__(s, config, verbose = 0): + UIBase.__init__(s, config, verbose) + s.safechars=" ;,./-_=+()[]" + s.iswaiting = 0 + s.outputlock = Lock() + s._printData('__init__', protocol) + + def isusable(s): + return True + + def _printData(s, command, data, dolock = True): + s._printDataOut('msg', command, data, dolock) + + def _printWarn(s, command, data, dolock = True): + s._printDataOut('warn', command, data, dolock) + + def _printDataOut(s, datatype, command, data, dolock = True): + if dolock: + s.outputlock.acquire() + try: + print "%s:%s:%s:%s" % \ + (datatype, + urllib.quote(command, s.safechars), + urllib.quote(currentThread().getName(), s.safechars), + urllib.quote(data, s.safechars)) + sys.stdout.flush() + finally: + if dolock: + s.outputlock.release() + + def _display(s, msg): + s._printData('_display', msg) + + def warn(s, msg, minor): + s._printData('warn', '%s\n%d' % (msg, int(minor))) + + def registerthread(s, account): + UIBase.registerthread(s, account) + s._printData('registerthread', account) + + def unregisterthread(s, thread): + UIBase.unregisterthread(s, thread) + s._printData('unregisterthread', thread.getName()) + + def debugging(s, debugtype): + s._printData('debugging', debugtype) + + def acct(s, accountname): + s._printData('acct', accountname) + + def acctdone(s, accountname): + s._printData('acctdone', accountname) + + def validityproblem(s, folder): + s._printData('validityproblem', "%s\n%s\n%s\n%s" % \ + (folder.getname(), folder.getrepository().getname(), + folder.getsaveduidvalidity(), folder.getuidvalidity())) + + def connecting(s, hostname, port): + s._printData('connecting', "%s\n%s" % (hostname, str(port))) + + def syncfolders(s, srcrepos, destrepos): + s._printData('syncfolders', "%s\n%s" % (s.getnicename(srcrepos), + s.getnicename(destrepos))) + + def syncingfolder(s, srcrepos, srcfolder, destrepos, destfolder): + s._printData('syncingfolder', "%s\n%s\n%s\n%s\n" % \ + (s.getnicename(srcrepos), srcfolder.getname(), + s.getnicename(destrepos), destfolder.getname())) + + def loadmessagelist(s, repos, folder): + s._printData('loadmessagelist', "%s\n%s" % (s.getnicename(repos), + folder.getvisiblename())) + + def messagelistloaded(s, repos, folder, count): + s._printData('messagelistloaded', "%s\n%s\n%d" % \ + (s.getnicename(repos), folder.getname(), count)) + + def syncingmessages(s, sr, sf, dr, df): + s._printData('syncingmessages', "%s\n%s\n%s\n%s\n" % \ + (s.getnicename(sr), sf.getname(), s.getnicename(dr), + df.getname())) + + def copyingmessage(s, uid, src, destlist): + ds = s.folderlist(destlist) + s._printData('copyingmessage', "%d\n%s\n%s\n%s" % \ + (uid, s.getnicename(src), src.getname(), ds)) + + def folderlist(s, list): + return ("\f".join(["%s\t%s" % (s.getnicename(x), x.getname()) for x in list])) + + def deletingmessage(s, uid, destlist): + s.deletingmessages(s, [uid], destlist) + + def uidlist(s, list): + return ("\f".join([str(u) for u in list])) + + def deletingmessages(s, uidlist, destlist): + ds = s.folderlist(destlist) + s._printData('deletingmessages', "%s\n%s" % (s.uidlist(uidlist), ds)) + + def addingflags(s, uidlist, flags, destlist): + ds = s.folderlist(destlist) + s._printData("addingflags", "%s\n%s\n%s" % (s.uidlist(uidlist), + "\f".join(flags), + ds)) + + def deletingflags(s, uidlist, flags, destlist): + ds = s.folderlist(destlist) + s._printData('deletingflags', "%s\n%s\n%s" % (s.uidlist(uidlist), + "\f".join(flags), + ds)) + + def threadException(s, thread): + print s.getThreadExceptionString(thread) + s._printData('threadException', "%s\n%s" % \ + (thread.getName(), s.getThreadExceptionString(thread))) + s.delThreadDebugLog(thread) + s.terminate(100) + + def terminate(s, exitstatus = 0, errortitle = '', errormsg = ''): + s._printData('terminate', "%d\n%s\n%s" % (exitstatus, errortitle, errormsg)) + sys.exit(exitstatus) + + def mainException(s): + s._printData('mainException', s.getMainExceptionString()) + + def threadExited(s, thread): + s._printData('threadExited', thread.getName()) + UIBase.threadExited(s, thread) + + def sleeping(s, sleepsecs, remainingsecs): + s._printData('sleeping', "%d\n%d" % (sleepsecs, remainingsecs)) + if sleepsecs > 0: + time.sleep(sleepsecs) + return 0 + + + def getpass(s, accountname, config, errmsg = None): + s.outputlock.acquire() + try: + if errmsg: + s._printData('getpasserror', "%s\n%s" % (accountname, errmsg), + False) + s._printData('getpass', accountname, False) + return (sys.stdin.readline()[:-1]) + finally: + s.outputlock.release() + + def init_banner(s): + s._printData('initbanner', offlineimap.version.banner) + diff --git a/offlineimap/ui/Noninteractive.py b/offlineimap/ui/Noninteractive.py index 18b18a7..c86a903 100644 --- a/offlineimap/ui/Noninteractive.py +++ b/offlineimap/ui/Noninteractive.py @@ -43,6 +43,9 @@ class Basic(UIBase): time.sleep(sleepsecs) return 0 + def locked(s): + s.warn("Another OfflineIMAP is running with the same metadatadir; exiting.") + class Quiet(Basic): def __init__(s, config, verbose = -1): Basic.__init__(s, config, verbose) diff --git a/offlineimap/ui/Tk.py b/offlineimap/ui/Tk.py deleted file mode 100644 index 2dc586e..0000000 --- a/offlineimap/ui/Tk.py +++ /dev/null @@ -1,543 +0,0 @@ -# Tk UI -# Copyright (C) 2002, 2003 John Goerzen -# -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA - -from __future__ import nested_scopes - -from Tkinter import * -import tkFont -from threading import * -import thread, traceback, time, threading -from StringIO import StringIO -from ScrolledText import ScrolledText -from offlineimap import threadutil, version -from Queue import Queue -from UIBase import UIBase -from offlineimap.ui.Blinkenlights import BlinkenBase - -usabletest = None - -class PasswordDialog: - def __init__(self, accountname, config, master=None, errmsg = None): - self.top = Toplevel(master) - self.top.title(version.productname + " Password Entry") - text = '' - if errmsg: - text = '%s: %s\n' % (accountname, errmsg) - text += "%s: Enter password: " % accountname - self.label = Label(self.top, text = text) - self.label.pack() - - self.entry = Entry(self.top, show='*') - self.entry.bind("", self.ok) - self.entry.pack() - self.entry.focus_force() - - self.button = Button(self.top, text = "OK", command=self.ok) - self.button.pack() - - self.entry.focus_force() - self.top.wait_window(self.label) - - def ok(self, args = None): - self.password = self.entry.get() - self.top.destroy() - - def getpassword(self): - return self.password - -class TextOKDialog: - def __init__(self, title, message, blocking = 1, master = None): - if not master: - self.top = Tk() - else: - self.top = Toplevel(master) - self.top.title(title) - self.text = ScrolledText(self.top, font = "Courier 10") - self.text.pack() - self.text.insert(END, message) - self.text['state'] = DISABLED - self.button = Button(self.top, text = "OK", command=self.ok) - self.button.pack() - - if blocking: - self.top.wait_window(self.button) - - def ok(self): - self.top.destroy() - - - -class ThreadFrame(Frame): - def __init__(self, master=None): - self.threadextraframe = None - self.thread = currentThread() - self.threadid = thread.get_ident() - Frame.__init__(self, master, relief = RIDGE, borderwidth = 2) - self.pack(fill = 'x') - self.threadlabel = Label(self, foreground = '#FF0000', - text ="Thread %d (%s)" % (self.threadid, - self.thread.getName())) - self.threadlabel.pack() - self.setthread(currentThread()) - - self.account = "Unknown" - self.mailbox = "Unknown" - self.loclabel = Label(self, - text = "Account/mailbox information unknown") - #self.loclabel.pack() - - self.updateloclabel() - - self.message = Label(self, text="Messages will appear here.\n", - foreground = '#0000FF') - self.message.pack(fill = 'x') - - def setthread(self, newthread): - if newthread: - self.threadlabel['text'] = newthread.getName() - else: - self.threadlabel['text'] = "No thread" - self.destroythreadextraframe() - - def destroythreadextraframe(self): - if self.threadextraframe: - self.threadextraframe.destroy() - self.threadextraframe = None - - def getthreadextraframe(self): - if self.threadextraframe: - return self.threadextraframe - self.threadextraframe = Frame(self) - self.threadextraframe.pack(fill = 'x') - return self.threadextraframe - - def setaccount(self, account): - self.account = account - self.mailbox = "Unknown" - self.updateloclabel() - - def setmailbox(self, mailbox): - self.mailbox = mailbox - self.updateloclabel() - - def updateloclabel(self): - self.loclabel['text'] = "Processing %s: %s" % (self.account, - self.mailbox) - - def appendmessage(self, newtext): - self.message['text'] += "\n" + newtext - - def setmessage(self, newtext): - self.message['text'] = newtext - - -class VerboseUI(UIBase): - def isusable(s): - global usabletest - if usabletest != None: - return usabletest - - try: - Tk().destroy() - usabletest = 1 - except TclError: - usabletest = 0 - return usabletest - - def _createTopWindow(self, doidlevac = 1): - self.notdeleted = 1 - self.created = threading.Event() - - self.af = {} - self.aflock = Lock() - - t = threadutil.ExitNotifyThread(target = self._runmainloop, - name = "Tk Mainloop") - t.setDaemon(1) - t.start() - - self.created.wait() - del self.created - - if doidlevac: - t = threadutil.ExitNotifyThread(target = self.idlevacuum, - name = "Tk idle vacuum") - t.setDaemon(1) - t.start() - - def _runmainloop(s): - s.top = Tk() - s.top.title(version.productname + " " + version.versionstr) - s.top.after_idle(s.created.set) - s.top.mainloop() - s.notdeleted = 0 - - def getaccountframe(s): - accountname = s.getthreadaccount() - s.aflock.acquire() - try: - if accountname in s.af: - return s.af[accountname] - - s.af[accountname] = LEDAccountFrame(s.top, accountname, - s.fontfamily, s.fontsize) - finally: - s.aflock.release() - return s.af[accountname] - - def getpass(s, accountname, config, errmsg = None): - pd = PasswordDialog(accountname, config, errmsg = errmsg) - return pd.getpassword() - - def gettf(s, newtype=ThreadFrame, master = None): - if master == None: - master = s.top - threadid = thread.get_ident() - s.tflock.acquire() - try: - if threadid in s.threadframes: - return s.threadframes[threadid] - if len(s.availablethreadframes): - tf = s.availablethreadframes.pop(0) - tf.setthread(currentThread()) - else: - tf = newtype(master) - s.threadframes[threadid] = tf - return tf - finally: - s.tflock.release() - - def _display(s, msg): - s.gettf().setmessage(msg) - - def threadExited(s, thread): - threadid = thread.threadid - s.tflock.acquire() - if threadid in s.threadframes: - tf = s.threadframes[threadid] - tf.setthread(None) - tf.setaccount("Unknown") - tf.setmessage("Idle") - s.availablethreadframes.append(tf) - del s.threadframes[threadid] - s.tflock.release() - UIBase.threadExited(s, thread) - - def idlevacuum(s): - while s.notdeleted: - time.sleep(10) - s.tflock.acquire() - while len(s.availablethreadframes): - tf = s.availablethreadframes.pop() - tf.destroy() - s.tflock.release() - - def terminate(s, exitstatus = 0, errortitle = None, errormsg = None): - if errormsg <> None: - if errortitle == None: - errortitle = "Error" - TextOKDialog(errortitle, errormsg) - UIBase.terminate(s, exitstatus = exitstatus, errortitle = None, errormsg = None) - - def threadException(s, thread): - exceptionstr = s.getThreadExceptionString(thread) - print exceptionstr - - s.top.destroy() - s.top = None - TextOKDialog("Thread Exception", exceptionstr) - s.delThreadDebugLog(thread) - s.terminate(100) - - def mainException(s): - exceptionstr = s.getMainExceptionString() - print exceptionstr - - s.top.destroy() - s.top = None - TextOKDialog("Main Program Exception", exceptionstr) - - def warn(s, msg, minor = 0): - if minor: - # Just let the default handler catch it - UIBase.warn(s, msg, minor) - else: - TextOKDialog("OfflineIMAP Warning", msg) - - def showlicense(s): - TextOKDialog(version.productname + " License", - version.bigcopyright + "\n" + - version.homepage + "\n\n" + version.license, - blocking = 0, master = s.top) - - - def init_banner(s): - s.threadframes = {} - s.availablethreadframes = [] - s.tflock = Lock() - s._createTopWindow() - s._msg(version.productname + " " + version.versionstr + ", " +\ - version.copyright) - tf = s.gettf().getthreadextraframe() - - b = Button(tf, text = "About", command = s.showlicense) - b.pack(side = LEFT) - - b = Button(tf, text = "Exit", command = s.terminate) - b.pack(side = RIGHT) - s.sleeping_abort = {} - - def deletingmessages(s, uidlist, destlist): - ds = s.folderlist(destlist) - s._msg("Deleting %d messages in %s" % (len(uidlist), ds)) - - def _sleep_cancel(s, args = None): - s.sleeping_abort[thread.get_ident()] = 1 - - def sleep(s, sleepsecs): - threadid = thread.get_ident() - s.sleeping_abort[threadid] = 0 - tf = s.gettf().getthreadextraframe() - - def sleep_cancel(): - s.sleeping_abort[threadid] = 1 - - sleepbut = Button(tf, text = 'Sync immediately', - command = sleep_cancel) - sleepbut.pack() - UIBase.sleep(s, sleepsecs) - - def sleeping(s, sleepsecs, remainingsecs): - retval = s.sleeping_abort[thread.get_ident()] - if remainingsecs: - s._msg("Next sync in %d:%02d" % (remainingsecs / 60, - remainingsecs % 60)) - else: - s._msg("Wait done; synchronizing now.") - s.gettf().destroythreadextraframe() - del s.sleeping_abort[thread.get_ident()] - time.sleep(sleepsecs) - return retval - -TkUI = VerboseUI - -################################################## Blinkenlights - -class LEDAccountFrame: - def __init__(self, top, accountname, fontfamily, fontsize): - self.top = top - self.accountname = accountname - self.fontfamily = fontfamily - self.fontsize = fontsize - self.frame = Frame(self.top, background = 'black') - self.frame.pack(side = BOTTOM, expand = 1, fill = X) - self._createcanvas(self.frame) - - self.label = Label(self.frame, text = accountname, - background = "black", foreground = "blue", - font = (self.fontfamily, self.fontsize)) - self.label.grid(sticky = E, row = 0, column = 1) - - def getnewthreadframe(s): - return LEDThreadFrame(s.canvas) - - def _createcanvas(self, parent): - c = LEDFrame(parent) - self.canvas = c - c.grid(sticky = E, row = 0, column = 0) - parent.grid_columnconfigure(1, weight = 1) - #c.pack(side = LEFT, expand = 0, fill = X) - - def startsleep(s, sleepsecs): - s.sleeping_abort = 0 - s.button = Button(s.frame, text = "Sync now", command = s.syncnow, - background = "black", activebackground = "black", - activeforeground = "white", - foreground = "blue", highlightthickness = 0, - padx = 0, pady = 0, - font = (s.fontfamily, s.fontsize), borderwidth = 0, - relief = 'solid') - s.button.grid(sticky = E, row = 0, column = 2) - - def syncnow(s): - s.sleeping_abort = 1 - - def sleeping(s, sleepsecs, remainingsecs): - if remainingsecs: - s.button.config(text = 'Sync now (%d:%02d remain)' % \ - (remainingsecs / 60, remainingsecs % 60)) - time.sleep(sleepsecs) - else: - s.button.destroy() - del s.button - return s.sleeping_abort - -class LEDFrame(Frame): - """This holds the different lights.""" - def getnewobj(self): - retval = Canvas(self, background = 'black', height = 20, bd = 0, - highlightthickness = 0, width = 10) - retval.pack(side = LEFT, padx = 0, pady = 0, ipadx = 0, ipady = 0) - return retval - -class LEDThreadFrame: - """There is one of these for each little light.""" - def __init__(self, master): - self.canvas = master.getnewobj() - self.color = '' - self.ovalid = self.canvas.create_oval(4, 4, 9, - 9, fill = 'gray', - outline = '#303030') - - def setcolor(self, newcolor): - if newcolor != self.color: - self.canvas.itemconfigure(self.ovalid, fill = newcolor) - self.color = newcolor - - def getcolor(self): - return self.color - - def setthread(self, newthread): - if newthread: - self.setcolor('gray') - else: - self.setcolor('black') - - -class Blinkenlights(BlinkenBase, VerboseUI): - def __init__(s, config, verbose = 0): - VerboseUI.__init__(s, config, verbose) - s.fontfamily = 'Helvetica' - s.fontsize = 8 - if config.has_option('ui.Tk.Blinkenlights', 'fontfamily'): - s.fontfamily = config.get('ui.Tk.Blinkenlights', 'fontfamily') - if config.has_option('ui.Tk.Blinkenlights', 'fontsize'): - s.fontsize = config.getint('ui.Tk.Blinkenlights', 'fontsize') - - def isusable(s): - return VerboseUI.isusable(s) - - def _createTopWindow(self): - VerboseUI._createTopWindow(self, 0) - #self.top.resizable(width = 0, height = 0) - self.top.configure(background = 'black', bd = 0) - - widthmetric = tkFont.Font(family = self.fontfamily, size = self.fontsize).measure("0") - self.loglines = self.config.getdefaultint("ui.Tk.Blinkenlights", - "loglines", 5) - self.bufferlines = self.config.getdefaultint("ui.Tk.Blinkenlights", - "bufferlines", 500) - self.text = ScrolledText(self.top, bg = 'black', #scrollbar = 'y', - font = (self.fontfamily, self.fontsize), - bd = 0, highlightthickness = 0, setgrid = 0, - state = DISABLED, height = self.loglines, - wrap = NONE, width = 60) - self.text.vbar.configure(background = '#000050', - activebackground = 'blue', - highlightbackground = 'black', - troughcolor = "black", bd = 0, - elementborderwidth = 2) - - self.textenabled = 0 - self.tags = [] - self.textlock = Lock() - - def init_banner(s): - BlinkenBase.init_banner(s) - s._createTopWindow() - menubar = Menu(s.top, activebackground = "black", - activeforeground = "white", - activeborderwidth = 0, - background = "black", foreground = "blue", - font = (s.fontfamily, s.fontsize), bd = 0) - menubar.add_command(label = "About", command = s.showlicense) - menubar.add_command(label = "Show Log", command = s._togglelog) - menubar.add_command(label = "Exit", command = s.terminate) - s.top.config(menu = menubar) - s.menubar = menubar - s.text.see(END) - if s.config.getdefaultboolean("ui.Tk.Blinkenlights", "showlog", 1): - s._togglelog() - s.gettf().setcolor('red') - s.top.resizable(width = 0, height = 0) - s._msg(version.banner) - - def _togglelog(s): - if s.textenabled: - s.oldtextheight = s.text.winfo_height() - s.text.pack_forget() - s.textenabled = 0 - s.menubar.entryconfig('Hide Log', label = 'Show Log') - s.top.update() - s.top.geometry("") - s.top.update() - s.top.resizable(width = 0, height = 0) - s.top.update() - - else: - s.text.pack(side = TOP, expand = 1, fill = BOTH) - s.textenabled = 1 - s.top.update() - s.top.geometry("") - s.menubar.entryconfig('Show Log', label = 'Hide Log') - s._rescroll() - s.top.resizable(width = 1, height = 1) - - def sleep(s, sleepsecs): - s.gettf().setcolor('red') - s._msg("Next sync in %d:%02d" % (sleepsecs / 60, sleepsecs % 60)) - BlinkenBase.sleep(s, sleepsecs) - - def sleeping(s, sleepsecs, remainingsecs): - return BlinkenBase.sleeping(s, sleepsecs, remainingsecs) - - def _rescroll(s): - s.text.see(END) - lo, hi = s.text.vbar.get() - s.text.vbar.set(1.0 - (hi - lo), 1.0) - - def _display(s, msg): - if "\n" in msg: - for thisline in msg.split("\n"): - s._msg(thisline) - return - #VerboseUI._msg(s, msg) - color = s.gettf().getcolor() - rescroll = 1 - s.textlock.acquire() - try: - if s.text.vbar.get()[1] != 1.0: - rescroll = 0 - s.text.config(state = NORMAL) - if not color in s.tags: - s.text.tag_config(color, foreground = color) - s.tags.append(color) - s.text.insert(END, "\n" + msg, color) - - # Trim down. Not quite sure why I have to say 7 instead of 5, - # but so it is. - while float(s.text.index(END)) > s.bufferlines + 2.0: - s.text.delete(1.0, 2.0) - - if rescroll: - s._rescroll() - finally: - s.text.config(state = DISABLED) - s.textlock.release() - - diff --git a/offlineimap/ui/UIBase.py b/offlineimap/ui/UIBase.py index fc1610c..d0e92e6 100644 --- a/offlineimap/ui/UIBase.py +++ b/offlineimap/ui/UIBase.py @@ -151,17 +151,17 @@ class UIBase: ################################################## WARNINGS def msgtoreadonly(s, destfolder, uid, content, flags): - if not (config.has_option('general', 'ignore-readonly') and config.getboolean("general", "ignore-readonly")): + if not (s.config.has_option('general', 'ignore-readonly') and s.config.getboolean("general", "ignore-readonly")): s.warn("Attempted to synchronize message %d to folder %s[%s], but that folder is read-only. The message will not be copied to that folder." % \ (uid, s.getnicename(destfolder), destfolder.getname())) def flagstoreadonly(s, destfolder, uidlist, flags): - if not (config.has_option('general', 'ignore-readonly') and config.getboolean("general", "ignore-readonly")): + if not (s.config.has_option('general', 'ignore-readonly') and s.config.getboolean("general", "ignore-readonly")): s.warn("Attempted to modify flags for messages %s in folder %s[%s], but that folder is read-only. No flags have been modified for that message." % \ (str(uidlist), s.getnicename(destfolder), destfolder.getname())) def deletereadonly(s, destfolder, uidlist): - if not (config.has_option('general', 'ignore-readonly') and config.getboolean("general", "ignore-readonly")): + if not (s.config.has_option('general', 'ignore-readonly') and s.config.getboolean("general", "ignore-readonly")): s.warn("Attempted to delete messages %s in folder %s[%s], but that folder is read-only. No messages have been deleted in that folder." % \ (str(uidlist), s.getnicename(destfolder), destfolder.getname())) @@ -208,6 +208,11 @@ class UIBase: s.getnicename(srcrepos), s.getnicename(destrepos))) + def skippingfolder(s, folder): + """Called when a folder sync operation is started.""" + if s.verbose >= 0: + s._msg("Skipping %s (not changed)" % folder.getname()) + def validityproblem(s, folder): s.warn("UID validity problem for folder %s (repo %s) (saved %d; got %d); skipping it" % \ (folder.getname(), folder.getrepository().getname(), diff --git a/offlineimap/ui/__init__.py b/offlineimap/ui/__init__.py index c1bdb2e..0206ab4 100644 --- a/offlineimap/ui/__init__.py +++ b/offlineimap/ui/__init__.py @@ -23,13 +23,6 @@ try: except ImportError: pass -try: - import Tkinter -except ImportError: - pass -else: - import Tk - try: import curses except ImportError: @@ -38,6 +31,7 @@ else: import Curses import Noninteractive +import Machine # Must be last import detector diff --git a/offlineimap/ui/detector.py b/offlineimap/ui/detector.py index a320d1e..4ec7503 100644 --- a/offlineimap/ui/detector.py +++ b/offlineimap/ui/detector.py @@ -19,9 +19,9 @@ import offlineimap.ui import sys -DEFAULT_UI_LIST = ('Tk.Blinkenlights', 'Tk.VerboseUI', - 'Curses.Blinkenlights', 'TTY.TTYUI', - 'Noninteractive.Basic', 'Noninteractive.Quiet') +DEFAULT_UI_LIST = ('Curses.Blinkenlights', 'TTY.TTYUI', + 'Noninteractive.Basic', 'Noninteractive.Quiet', + 'Machine.MachineUI') def findUI(config, chosenUI=None): uistrlist = list(DEFAULT_UI_LIST) diff --git a/offlineimap/version.py b/offlineimap/version.py index eee6619..0713fad 100644 --- a/offlineimap/version.py +++ b/offlineimap/version.py @@ -1,11 +1,11 @@ productname = 'OfflineIMAP' -versionstr = "4.0.16" +versionstr = "5.99.4" versionlist = versionstr.split(".") major = versionlist[0] minor = versionlist[1] patch = versionlist[2] -copyright = "Copyright (C) 2002 - 2006 John Goerzen" +copyright = "Copyright (C) 2002 - 2007 John Goerzen" author = "John Goerzen" author_email = "jgoerzen@complete.org" description = "Disconnected Universal IMAP Mail Synchronization/Reader Support" @@ -18,7 +18,7 @@ COPYING for details. This is free software, and you are welcome to distribute it under the conditions laid out in COPYING.""" homepage = "http://software.complete.org/offlineimap/" -license = """Copyright (C) 2002 - 2006 John Goerzen +license = """Copyright (C) 2002 - 2007 John Goerzen This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -34,10 +34,16 @@ You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA""" -cmdhelp = """ +def getcmdhelp(): + from offlineimap.ui import detector + import os + uilist = "" + for ui in detector.DEFAULT_UI_LIST: + uilist += " " + ui + os.linesep + return """ offlineimap [ -1 ] [ -P profiledir ] [ -a accountlist ] [ -c configfile ] [ -d debugtype[,debugtype...] ] [ -o ] [ - -u interface ] + -u interface ] [ -q ] offlineimap -h | --help @@ -88,9 +94,27 @@ cmdhelp = """ one else. The maildir option will enable debugging for certain Maildir operations. - -o Run only once, ignoring any autorefresh setting in + -f foldername[,foldername...] + Only sync the specified folders. The "foldername"s + are the *untranslated* foldernames. This + command-line option overrides any "folderfilter" + and "folderincludes" options in the configuration + file. + + -k [section:]option=value + Override configuration file option. If"section" is + omitted, it defaults to "general". Any underscores + "_" in the section name are replaced with spaces: + for instance, to override option "autorefresh" in + the "[Account Personal]" section in the config file + one would use "-k Account_Personal:autorefresh=30". + + -o Run only once, ignoring any autorefresh setting in the config file. + -q Run only quick synchronizations. Ignore any flag + updates on IMAP servers. + -h, --help Show summary of options. @@ -102,8 +126,4 @@ cmdhelp = """ states that it cannot be. Use this option with care. The pre-defined options, described in the USER INTERFACES section of the man page, are: -""" -from offlineimap.ui import detector -import os -for ui in detector.DEFAULT_UI_LIST: - cmdhelp += " " + ui + os.linesep +""" + uilist