In this tutorial we shall show you how easy it is to make simple real-time backup software by using the Python module called Pyinotify. This module is based on the Linux kernel feature named inotify (included since kernel 2.6.13); and this being an event-driven notifier, it’ll convey its notifications from kernel space to user space by means of using system-calls. The Pyinotify module will eventually bind such calls.

Let’s get to the code now: create a file labelled “backup.py” in your working directory and paste in the following code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154

import os, os.path, time, signal, sys, operator, glob, shutil, filecmp
from pyinotify import WatchManager, Notifier, ThreadedNotifier, EventsCodes, ProcessEvent
 
 
BACKUP = {
        "/home/user/my_dir" : "/home/user/my_backup",
        "/home/user/test" : "/home/user/test_backup"        
}
 
 
class FileSystemEvent(ProcessEvent):
 
        def __init__(self, watcher):
                self.f = FileBackup(watcher)                
 
        def process_event(self, event):
                self.f.process_file(event.path, event.name)
 
        def process_IN_CREATE(self, event):
                self.process_event(event)
 
        def process_IN_DELETE(self, event):
                self.process_event(event)
 
        def process_IN_MODIFY(self, event):
                self.process_event(event)
 
 
class FileBackup:
        def __init__(self, watcher):
                self.watcher = watcher
                self.l = BACKUP.keys()
 
        def process_file(self, event_path, event_name):
                path = os.path.join(event_path, event_name)
 
                if os.path.isdir(path):
                        self.watcher.add(path) #watch new dir
                        base = self.get_base_path(path)
                else:
                        base = self.get_base_path(os.path.dirname(path))
 
                dst = BACKUP[base[0]]+base[1]
 
                try:
                        os.makedirs(dst)
                except: pass
 
 
                if os.path.isfile(path):
                        if os.path.isfile(dst+"/"+event_name):
                                if filecmp.cmp(path, dst+"/"+event_name):
                                        return                                
 
                        shutil.copy2(path, dst+"/"+event_name)
 
 
        def get_base_path(self, path):                
                for p in self.l:
                        if (len(path.replace(p, '')) < len(path)):
                                return p, path.replace(p, '')
                return "",""
 
 
class FileSystemWatcher():
        def __init__(self, dirs=[]):
                self.wm       = WatchManager()
                self.fse      = FileSystemEvent(self)
 
                self.mask     = EventsCodes.IN_DELETE | EventsCodes.IN_CREATE | EventsCodes.IN_MODIFY # events
                self.notifier = ThreadedNotifier(self.wm, self.fse)
 
                for dir in dirs: self.add(dir)               
 
        def start(self):
                self.notifier.start()
 
        def block(self):
                self.notifier.join() 
 
        def close(self):
                self.notifier.close() 
 
        def add(self, dirpath):
                if os.path.isdir(dirpath):
                        self.wdd = self.wm.add_watch(dirpath, self.mask, rec=True)
 
 
class KeyboardCatcher:
 
        def __init__(self):
                self.child = os.fork()
 
                if self.child == 0:
                        return
                else:
                        self.watch()
 
        def watch(self):
                try:
                        os.wait()
 
                except KeyboardInterrupt:
                        print 'KeyBoardInterrupt'
                        self.kill()
                sys.exit()                               
 
        def kill(self):
                try:
                        os.kill(self.child, signal.SIGKILL)
                except OSError: pass
 
 
 
 
def _copy_base_dir(src, dst):
        try:
                os.makedirs(dst)
 
        except OSError:                
                pass # Directory already exists!
 
        for path in glob.glob("%s/*" % src):
                dst2 = os.path.join(dst, os.path.basename(path))
 
                if os.path.isdir(path):
                        _copy_base_dir(path, dst2)                        
 
                elif os.path.isfile(path):
                        if os.path.isfile(dst2):
                                if filecmp.cmp(path, dst2):
                                        continue
 
                        shutil.copy2(path, dst)
 
 
def copy_base_dir():
        for src in BACKUP: 
                _copy_base_dir(src, BACKUP[src])        
 
 
 
def main():         
        KeyboardCatcher()
 
        #copy_base_dir()
 
        w = FileSystemWatcher(BACKUP.keys())
        w.start()          
        w.block()
 
 
if __name__ == "__main__":
        main()

With this script we’ll be able to backup our files in real-time.
First off we created a dictionary called BACKUP, in which we set those directories which are to be monitored and where we’re going to save all our files; in this way we can back-up one or more directories.

BACKUP = {
        "from1" : "to1",
        "from2" : "to2"
}

Now… on with the main() function:

The first instruction, KeyboardCatcher(), provides us with a new process in our program. This class is not mandatory, but allows us to catch any possible keyboard exceptions (ctrl-c). The main program will stand by, waiting; whilst the child process can continue following the script.

We commented out the call to the copy_base_dir() method, but you could use it as best suits your necessities. In a nutshell this method will back-up the directories—before starting with the file-system monitoring—and skip every already-existing file.

We reach now the program’s very core, that is monitoring the file-system using python.
We’ve done a class called FileSystemWatcher to which we pass the list of files to be monitored. This class has the office of ‘listening’ for file-system events via the Pyinotify module.

Of course, we’re the ones to appoint which events are to be captured. Indeed, with the following line, we choose the events we want secured.

self.mask = EventsCodes.IN_DELETE | EventsCodes.IN_CREATE | EventsCodes.IN_MODIFY

Establishing the events’ names is not enough though, we’ve to make a class whose callback functions are triggered by the occurrence of such event; in this case it’s the class FileSystemEvent with its three callbacks: process_IN_CREATE – process_IN_DELETE – process_IN_MODIFY.

Finally the class FileBackup, thanks to the process_file method, will check whether the file to be saved is already in the backup directory.

As previously addressed, this is a very simple script to make backups and could be surely improved and optimized.

Well, have fun!

NOTE: The script has been tested with Python 2.5.2 and, needless to say, with GNU Linux OS; those of you who are MS-windows dependent should probably fast for this time

I bet many of you have wondered, at least once, how much time your method or class is going to take when executed. Well, thanks to the python timeit module we can really time our own code.
Quite often when tweaking with the code, it would be quite a good practice to test such tweaks by means of timing them; in this way we could be able to see which tweaked version of the code takes the more time. In this tutorial we’re going into the timeit module which has been added since python 2.3.

Using the aforementioned module is definitely simple; indeed with just three lines of code we shall be able to assess the execution time of our script, or part of it that is.

Let’s create a file called “test_timeit.py” in our working directory and let’s also copy the following code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

import timeit
 
class Tester:
        def __init__(self):
                self.MAX = 100
                self.my_list = []
 
 
        def add_to_list(self):
                for i in range(self.MAX):
                        self.my_list.append(i)
 
 
        def remove_from_list(self):
                for elem in self.my_list:
                        self.my_list.remove(elem)
 
 
if __name__=="__main__":
        inst = Tester()
        t1   = timeit.Timer("inst.add_to_list()", "from __main__ import inst")
	#t1  = timeit.Timer("inst.remove_from_list()", "from __main__ import inst")
 
        try:
                print t1.timeit(200)        
		print t1.repeat(repeat=3, number=200)
        except:
                t1.print_exc()

Now for the explanatory part:

First of all we created a simple class named “Tester”, thus bundling two methods which we shall use to determine the execution time; they’re called add_to_list() and remove_to_list() and they’ll take care of adding or removing elements from our list, named as “my_list”.

Let’s get to the __main__ part now where, thanks to inst = Tester(), we shall create a new instance of our Tester class—this instance will let us reach each method which has to be timed.
Then from line 21 onwards, we tackle the timeit part.

The first one:

t1 = timeit.Timer("inst.add_to_list()", "from __main__ import inst")

by which we create an instance of the Timer class that will let us measure the execution time of our python code—each parameter used in this example is referring respectively to the method we time, namely “inst.add_to_list()”, and to the mandatory import which is enabling us to get an instance called inst.

NOTE: We would get the same result with: t1 = timeit.Timer(”Tester().add_to_list()”, “from __main__ import Tester”)

Now, getting to the real time-appraising part, the Timer class deploys two methods:

timeit([number=1000000])

This method returns the execution time of the main statement, which has been recalled as many times as parameter “number” dictates; in our example(line 25) we’ve used print t1.timeit(200), that is to say, the add_to_list() method of Tester class will be called for 200 times, at the end of which the amount of time(float) spent on such office will be returned.

repeat([repeat=3[, number=1000000]])

The repeat() method works in the same way of timeit(), just with a trifling variation; quite useful.
The number specified by “repeat” pinpoints how many times we take the measure.
As it is in our example, print t1.repeat(repeat=3, number=200), the method add_to_list() of Tester class is being recalled 200 times, at the end of which a new measure ensues, and so on as many times as it is indicated in the number “repeat”. As we can easily guess, the repeat() method returns a list with every recorded amount of time.

Here come other two simple and explanatory examples:

#Example 1
def factorial(x):
	if x == 0:
		return 1
	else:
		return x * factorial(x-1)
 
if __name__=="__main__":
    from timeit import Timer
    t = Timer("factorial(5)", "from __main__ import factorial")
    print t.timeit()
 
 
#Example 2
import timeit
 
s = """
for i in range(10):
    if i % 2 == 0:
        pass
"""
t= timeit.Timer(s)
print t.timeit()

As you can clearly see, using timeit is that easy, with ever so few steps we were able to time our python code.
Well? What are you waiting for? Get on with your testing!

created by Damiano Porta and translated by Daniele Feuli

Today we will look at a method of using the SMTP protocol via Python. Our goal is to create a class through which we can send emails with attachments.

First, a word about the SMTP protocol.

• SMTP (Simple Mail Transfer Protocol) is the standard protocol for sending email via internet. By default, SMTP uses port number 25 and the transmission protocol is TCP. One of the first SMTP servers is sendmail, still widely in use, others exist, such as Postfix, Exim.

Ok, on to the code. The main way of sending emails via python is the smtplib.

Create a file called “smtp.py” in your working directory and paste the following code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113

import smtplib, os, time, atexit
from email.MIMEMultipart import MIMEMultipart
from email.MIMEBase import MIMEBase
from email.MIMEText import MIMEText
from email.Utils import formatdate
from email import Encoders
 
class ConnectionError(smtplib.SMTPException): pass
class LoginError(smtplib.SMTPException): pass
class DisconnectionError(smtplib.SMTPException): pass
class EmailSendError(smtplib.SMTPException): pass
 
class Smtp:
 
        def __init__(self, host, user, password, port=25):
                self._host        = host
                self._port        = port                
                self._user        = user
                self._password    = password
 
                self._message     = None
                self._subject     = None
                self._from_addr   = None
                self._rcpt_to     = None               
                self._server      = None         
                self._attachments = []
 
                atexit.register(close) #our close() method will be automatically executed upon normal interpreter termination
 
                self.connect()
 
 
        def connect(self):
 
                if all([self._host, self._port, self._user, self._password]):
 
                        try:    
                                self._server = smtplib.SMTP(self._host, self._port)
 
                        except smtplib.SMTPException, e:
                                raise ConnectionError("Connection failed!")
 
                        try:
                                self._server.login(self._user, self._password)        
 
                        except smtplib.SMTPException, e:                                
                                raise LoginError("Login Failed!")
 
 
 
        def close(self):                
 
                if self._server:
                        try:
                                self._server.quit()
 
                        except smtplib.SMTPException, e:                                
                                raise DisconnectionError("Disconnection failed!")
 
 
        def message(self, message):
                self._message = message
 
 
        def subject(self, subject):
                self._subject = subject
 
 
        def from_addr(self, email):
                self._from_addr = email
 
 
        def rcpt_to(self, email):
                self._rcpt_to = email
 
 
        def attach(self, file):
                if os.path.exists(file):
                        self._attachments.append(file)
 
 
        def load_attachments(self, m_message):
                for file in self._attachments:
                        part = MIMEBase('application', "octet-stream")
                        part.set_payload(open(file,"rb").read())
                        Encoders.encode_base64(part)
                        part.add_header('Content-Disposition', 'attachment; filename="%s"' % os.path.basename(file))
                        m_message.attach(part)    
 
                return m_message
 
 
        def send(self, content_type='plain', charset='UTF-8'):
 
                if all([self._message, self._subject, self._from_addr, self._rcpt_to]):                                
 
                        m_message             = MIMEMultipart()
 
                        m_message['From']     = self._from_addr
                        m_message['To']       = self._rcpt_to
                        m_message['Date']     = formatdate(localtime=True)
                        m_message['Subject']  = self._subject
                        m_message['X-Mailer'] = "Python X-Mailer"
 
                        m_message.attach(MIMEText(self._message, content_type, charset))
 
                        m_message = self.load_attachments(m_message)
 
                        try:
                                self._server.sendmail(self._from_addr, self._rcpt_to, m_message.as_string())       
 
                        except smtplib.SMTPException, e:
                                raise EmailSendError("Email has not been sent")

Here we have a simple class ready for sending email over an authenticated connection via python.

Now for the explanation (This should be easy to understand for those of us who already use python a bit).

Skipping over the various module import commands required to run our class, we come to the declaration of 4 classes (from line 8 to 11) needed for error management.

• ConnectionError – the exception generated if there is a problem in connecting to the SMTP server.
• Loginerror – the exception generated when there is a problem with the login (user/password).
• DisconnectionError – the exception generated when there is a problem when disconnecting from SMTP server.
• EmailSendError – the exception generated when there is a problem in sending the email.

As you can see every exception inherits smtplib.SMTPException. In this way you can also catch our exceptions, by using the baseclass smtplib module.

Now we move on to the explanation of SMTP class…

The class has an initialiser for the declaration of all attributes for the class and automatically calls the connect() method to connect to the SMTP server.

Moving on… at line number 33, we meet our connect() method, in which we will create an instance of the SMTP class (smtplib) and will then try to authenticate to the smtp server with self._server.login(). If there are errors in the connection or the login to the server, the two exceptions (ConnectionError, Loginerror) will be generated.

To successfully send an email certain data are needed: the sender, recipient, subject and message. For this reason, using our methods: from_addr(), rcpt_to(), subject(), and message() we will be able to upload all data needed to send our email. In addition to these basic methods, our class has a very useful method for attaching a file or files to our email, attach(file_path).

Now lets give it a try! Create a file named “test.py” in your working directory and paste the following code:

from smtp import Smtp
 
smtp = Smtp("mail.server.com", "user", "password")
smtp.subject('This is a test')
smtp.message('Email message')
smtp.from_addr('from@server.com')
smtp.rcpt_to('to@server.com')
smtp.attach("file1.jpg")
smtp.attach("file2.gif")
smtp.attach("file3.pdf")
smtp.send()

Need more explanation?

Have fun using this!

The Python Imaging Libray (PIL) is a library that allows Python Programmers to open, save, and manipulate images. PIL mostly deals with image formats TIFF, JPG, BMP, PNG, and GIF, but there are even ways for a common user to add their own formats to the Library. The Python Imaging Library greatly speeds up programming time if the Python program deals with images at all.

To open an image with PIL, the library module merely needs to be defined, and then the Open command needs to be issued. To create an image from scratch, the “new” command must be used. Saving an image is as simple as using the “save” command. (Use “save” after modifying or creating an image).

#start out python code with the import of the Image Library
#The Image and ImageDraw libraries are part of PIL
 
import Image, ImageDraw
 
#now open the mypicture.bmp image, assign the image to the variable "bmp"
 
bmp = Image.open("mypicture.bmp")
 
#create a new image using the "new" command
 
newbmp = Image.new("RGB", (128, 128), "White")
 
#save the new image
 
newbmp.save("mypicture2.bmp")
 
#now, display both of the images
 
bmp.show()
 
newbmp.show()
 
# To do more than just a block of color, there are simple commands to write whatever you want.
# This is where the beauty of PIL comes into play.
# To draw a red line on the black image that was just created
 
draw = ImageDraw.Draw(newbmp)
 
draw.line((0, 0, 100, 100), fill=rgb(255,0,0))
 
del draw
 
# And to save and see the recent handiwork
 
newbmp.save("mypicturewithline.bmp")
 
newbmp.show()

Using PIL, you can also grab a single pixel and change it. Let’s see hows that’s done…

import Image, ImageDraw
 
bmp = Image.open("mypicture.bmp")
 
draw = ImageDraw.Draw(bmp )
 
draw.point((64,64), fill=None)
 
del draw
 
bmp.show()

To be able to work with PIL, you only need to know a few functions. New and open will get you an image to work with, save will write your image to the disk, and show will display what you’ve done so far. A bunch of other functions exist for modifying your image once you have it open. You can draft, convert (from one file type to another), crop (cut the image to a new size), and thumbnail and resize (excellent for automatic blogging or news sites).

How to add a watermark to your images using Python Imaging Library

One method involves putting one image on top of another. You will need an image of your transparent watermark handy, and then you’ll need the image you want to put it on. Then it’s just a simple process of “pasting” your watermark on the image. Let’s see how easy this process is using PIL.

import Image, ImageDraw
 
bmp= Image.open("mypicture.bmp")
 
mywatermark = Image.open("mywatermark.bmp")
 
bmp.paste(mywatermark, (0,0,100,100))
 
bmp.show()

Another water mark method just writes a text onto your image. You’ll need a little more doing, but it’s still really easy. (You will need a font file handy, just so you know.) In the middle of this function, you will notice a while loop. What we are doing here is testing the text size against the size of the image we want to watermark. We want the text to be a big as possible, and still remain inside the image. A little dynamic code goes a long way.

import Image, ImageDraw, ImageFont
 
bmp = Image.open("mypicture.bmp")
 
mywatermark = Image.new("RGBA", (bmp.size[0], bmp.size[1]))
 
drawing = ImageDraw.ImageDraw(mywatermark, "RGBA")
 
size = 0
 
while True:
 
    size += 1
 
    testfont = ImageFont.truetype(FONT, size)
 
    testtextw, testtexth = testfont.getsize("My Watermark Text")
 
    if testtextw+testtexth/3 > mywatermark.size[0]:
 
        break
 
    font = nextfont
 
    textw, texth = testtextw, testtexth
 
drawing.setfont(font)
 
drawing.text(((mywatermark.size[0]-textw)/2, (mywatermark.size[1]-texth)/2), "My Watermark Text")
 
bmp.paste(mywatermark, None, mywatermark)
 
bmp.show()

Nearly any image manipulation is incredibly simple when you use the Python Imaging Library. This is a very well thought out program that is freely available for you to use.