Programming Python (118 page)

It is possible to
extend this remote tree-cleaner to also download a remote
tree with subdirectories: rather than deleting, as you walk the remote
tree simply create a local directory to match a remote one, and download
nondirectory files. We’ll leave this final step as a suggested exercise,
though, partly because its dependence on the format produced by server
directory listings makes it complex to be robust and partly because this
use case is less common for me—in practice, I am more likely to maintain
a site on my PC and upload to the server than to download a tree.

If you do wish to experiment with a recursive download, though, be
sure to consult the script
Tools\Scripts\ftpmirror.py
in Python’s install or
source tree for hints. That script attempts to download a remote
directory tree by FTP, and allows for various directory listing formats
which we’ll skip here in the interest of space. For our purposes, it’s
time to move on to the next protocol on our tour—Internet
email.

• poplib
and
  imaplib
for
  fetching email
  

• smtplib
for sending
  email
  

• The
  email
module package for
  parsing email and constructing email
  

In the prior sections of this
chapter, we studied how Unicode encodings can impact
scripts using Python’s
ftplib
FTP
tools in some depth, because it illustrates the implications of Python
3.X’s Unicode string model for real-world programming. In short:

The prior sections describe why these rules are in force. The last
two points here differ for scripts written originally for Python 2.X. As
you might expect, given that the underlying sockets transfer byte
strings today, the email story is somewhat convoluted for Unicode in
Python 3.X as well. As a brief preview:

If you’re migrating email scripts (or your mindset) from 2.X,
you’ll need to treat email text fetched from a server as byte strings,
and encode it before passing it along for parsing; scripts that send or
compose email are generally unaffected (and this may be the majority of
Python email-aware scripts), though content may have to be treated
specially if it may be returned as byte strings.

This is the story in Python 3.1, which is of course prone to
change over time. We’ll see how these email constraints translate into
code as we move along in this section. Suffice it to say, the text on
the Internet is not as simple as it used to be, though it probably
shouldn’t have been
anyhow.

Before we get to the
scripts, let’s first take a look at a common module they
import and use. The module in
Example 13-17
is used to configure
email parameters appropriately for a particular user. It’s simply a
collection of assignments to variables used by mail programs that appear
in this book; each major mail client has its own version, to allow
content to vary. Isolating these configuration settings in this single
module makes it easy to configure the book’s email programs for a
particular user, without having to edit actual program logic
code.

If you want to use any of this book’s email programs to do mail
processing of your own, be sure to change its assignments to reflect
your servers, account usernames, and so on (as shown, they refer to
email accounts used for developing this book). Not all scripts use all
of these settings; we’ll revisit this module in later examples to
explain more of them.

Note that some ISPs may require that you be connected directly to
their systems in order to use their SMTP servers to send mail. For
example, when connected directly by dial-up in the past, I could use my
ISP’s server directly, but when connected via broadband, I had to route
requests through a cable Internet provider. You may need to adjust these
settings to match your configuration; see your ISP to obtain the
required POP and SMTP servers. Also, some SMTP servers check domain name
validity in addresses, and may require an authenticating login step—see
the SMTP section later in this chapter for interface details.

"""
user configuration settings for various email programs (pymail/mailtools version);
email scripts get their server names and other email config options from this
module: change me to reflect your server names and mail preferences;
"""
#------------------------------------------------------------------------------
# (required for load, delete: all) POP3 email server machine, user
#------------------------------------------------------------------------------
popservername = 'pop.secureserver.net'
popusername   = '[email protected]'
#------------------------------------------------------------------------------
# (required for send: all) SMTP email server machine name
# see Python smtpd module for a SMTP server class to run locally;
#------------------------------------------------------------------------------
smtpservername = 'smtpout.secureserver.net'
#------------------------------------------------------------------------------
# (optional: all) personal information used by clients to fill in mail if set;
# signature  -- can be a triple-quoted block, ignored if empty string;
# address -- used for initial value of "From" field if not empty,
# no longer tries to guess From for replies: this had varying success;
#------------------------------------------------------------------------------
myaddress   = '[email protected]'
mysignature = ('Thanks,\n'
'--Mark Lutz  (http://learning-python.com, http://rmi.net/~lutz)')
#------------------------------------------------------------------------------
# (optional: mailtools) may be required for send; SMTP user/password if
# authenticated; set user to None or '' if no login/authentication is
# required; set pswd to name of a file holding your SMTP password, or
# an empty string to force programs to ask (in a console, or GUI);
#------------------------------------------------------------------------------
smtpuser  = None                           # per your ISP
smtppasswdfile  = ''                       # set to '' to be asked
#------------------------------------------------------------------------------
# (optional: mailtools) name of local one-line text file with your pop
# password; if empty or file cannot be read, pswd is requested when first
# connecting; pswd not encrypted: leave this empty on shared machines;
#------------------------------------------------------------------------------
poppasswdfile  = r'c:\temp\pymailgui.txt'      # set to '' to be asked
#------------------------------------------------------------------------------
# (required: mailtools) local file where sent messages are saved by some clients;
#------------------------------------------------------------------------------
sentmailfile   = r'.\sentmail.txt'             # . means in current working dir
#------------------------------------------------------------------------------
# (required: pymail, pymail2) local file where pymail saves pop mail on request;
#------------------------------------------------------------------------------
savemailfile   = r'c:\temp\savemail.txt'       # not used in PyMailGUI: dialog
#------------------------------------------------------------------------------
# (required: pymail, mailtools) fetchEncoding is the Unicode encoding used to
# decode fetched full message bytes, and to encode and decode message text if
# stored in text-mode save files; see Chapter 13 for details: this is a limited
# and temporary approach to Unicode encodings until a new bytes-friendly email
# package is developed; headersEncodeTo is for sent headers: see chapter13;
#------------------------------------------------------------------------------
fetchEncoding = 'utf8'      # 4E: how to decode and store message text (or latin1?)
headersEncodeTo = None      # 4E: how to encode non-ASCII headers sent (None=utf8)
#------------------------------------------------------------------------------
# (optional: mailtools) the maximum number of mail headers or messages to
# download on each load request; given this setting N, mailtools fetches at
# most N of the most recently arrived mails; older mails outside this set are
# not fetched from the server, but are returned as empty/dummy emails; if this
# is assigned to None (or 0), loads will have no such limit; use this if you
# have very many mails in your inbox, and your Internet or mail server speed
# makes full loads too slow to be practical;  some clients also load only
# newly-arrived emails, but this setting is independent of that feature;
#------------------------------------------------------------------------------
fetchlimit = 25             # 4E: maximum number headers/emails to fetch on loads

On to
reading email in Python: the script in
Example 13-18
employs Python’s
standard
poplib
module, an
implementation of the client-side interface to POP—the Post Office
Protocol. POP is a well-defined and widely available way to fetch email
from servers over sockets. This script connects to a POP server to
implement a simple yet portable email download and display tool.

#!/usr/local/bin/python
"""
##############################################################################
use the Python POP3 mail interface module to view your POP email account
messages;  this is just a simple listing--see pymail.py for a client with
more user interaction features, and smtpmail.py for a script which sends
mail;  POP is used to retrieve mail, and runs on a socket using port number
110 on the server machine, but Python's poplib hides all protocol details;
to send mail, use the smtplib module (or os.popen('mail...')).  see also:
imaplib module for IMAP alternative, PyMailGUI/PyMailCGI for more features;
##############################################################################
"""
import poplib, getpass, sys, mailconfig
mailserver = mailconfig.popservername      # ex: 'pop.rmi.net'
mailuser   = mailconfig.popusername        # ex: 'lutz'
mailpasswd = getpass.getpass('Password for %s?' % mailserver)
print('Connecting...')
server = poplib.POP3(mailserver)
server.user(mailuser)                      # connect, log in to mail server
server.pass_(mailpasswd)                   # pass is a reserved word
try:
print(server.getwelcome())             # print returned greeting message
msgCount, msgBytes = server.stat()
print('There are', msgCount, 'mail messages in', msgBytes, 'bytes')
print(server.list())
print('-' * 80)
input('[Press Enter key]')
for i in range(msgCount):
hdr, message, octets = server.retr(i+1)    # octets is byte count
for line in message: print(line.decode())  # retrieve, print all mail
print('-' * 80)                            # mail text is bytes in 3.x
if i < msgCount - 1:
input('[Press Enter key]')              # mail box locked till quit
finally:                                           # make sure we unlock mbox
server.quit()                                  # else locked till timeout
print('Bye.')

Though primitive, this script illustrates the basics of reading
email in Python. To establish a connection to an email server, we start
by making an instance of the
poplib.POP3
object, passing in the email
server machine’s name as a string:

server = poplib.POP3(mailserver)

If this call doesn’t raise an exception, we’re connected (by
socket) to the POP server listening on POP port number 110 at the
machine where our email account lives.

The next thing we need to do before fetching messages is tell the
server our username and password; notice that the password method is
called
pass_
. Without the trailing
underscore,
pass
would name a
reserved word and trigger a syntax error:

server.user(mailuser)                      # connect, log in to mail server
server.pass_(mailpasswd)                   # pass is a reserved word

To keep things simple and relatively secure, this script always
asks for the account password interactively; the
getpass
module we met in the FTP section of
this chapter is used to input but not display a password string typed by
the user.

Once we’ve told the server our username and password, we’re free
to fetch mailbox information with the
stat
method (number messages, total bytes
among all messages) and fetch the full text of a particular message with
the
retr
method (pass the message
number—they start at 1). The full text includes all headers, followed by
a blank line, followed by the mail’s text and any attached parts. The
retr
call sends back a tuple that
includes a list of line strings representing the content of the
mail:

msgCount, msgBytes = server.stat()
hdr, message, octets = server.retr(i+1)    # octets is byte count

We close the email server connection by calling the POP object’s
quit
method:

server.quit()                              # else locked till timeout

Notice that this call appears inside the
finally
clause of a
try
statement that wraps the bulk of the
script. To minimize complications associated with changes, POP servers
lock your email inbox between the time you first connect and the time
you close your connection (or until an arbitrary, system-defined timeout
expires). Because the POP
quit
method
also unlocks the mailbox, it’s crucial that we do this before exiting,
whether an exception is raised during email processing or not. By
wrapping the action in a
try
/
finally
statement, we guarantee that the
script calls
quit
on exit to unlock
the mailbox to make it accessible to other processes (e.g., delivery of
incoming email).