]> code.delx.au - offlineimap/commitdiff
[319117] Unroll loop to speed performance on large folders
authorJohn Goerzen <jgoerzen@complete.org>
Tue, 23 Aug 2005 07:15:09 +0000 (08:15 +0100)
committerJohn Goerzen <jgoerzen@complete.org>
Tue, 23 Aug 2005 07:15:09 +0000 (08:15 +0100)
From: "Nikita V. Youshchenko"
I trued to use offlineimap and found that while being quite fast on
small folders, it takes up to several minutes (of 100% busy CPU and
almost no network traffic) to sync a folder with 2000+ messages.

While looking into the code, I found why this happens.
In folder/Base.py, in method BaseFolder.syncmessagesto_copy(),
dest.getmessagelist() is called inside a loop, while being a loop
invariant. Similar thing happens in BaseFolder.syncmessagesto_delete()
for self.getmessagelist().
This causes quadratic complexity over folder size.

Moving these calls out of loops make large folder sync fast (several
seconds instead of several minutes for folder with 2000 messages on
700MHz P3).

offlineimap/folder/Base.py

index 8d1bbae821ac51e6487a5b03dfed96e5cf2e4440..50d9ccbdd4e5cda79f79168048fa26f04f58339f 100644 (file)
@@ -288,10 +288,11 @@ class BaseFolder:
         them to dest."""
         threads = []
         
+       dest_messagelist = dest.getmessagelist()
         for uid in self.getmessagelist().keys():
             if uid < 0:                 # Ignore messages that pass 1 missed.
                 continue
-            if not uid in dest.getmessagelist():
+            if not uid in dest_messagelist:
                 if self.suggeststhreads():
                     self.waitforthread()
                     thread = InstanceLimitedThread(\
@@ -314,10 +315,11 @@ class BaseFolder:
         Look for message present in dest but not in self.
         If any, delete them."""
         deletelist = []
+       self_messagelist = self.getmessagelist()
         for uid in dest.getmessagelist().keys():
             if uid < 0:
                 continue
-            if not uid in self.getmessagelist():
+            if not uid in self_messagelist:
                 deletelist.append(uid)
         if len(deletelist):
             UIBase.getglobalui().deletingmessages(deletelist, applyto)