Programming Python (165 page)

I initially wrote the
send script to be used only within PyMailCGI using values
typed into the mail edit form. But as we’ve seen, inputs can be sent in
either form fields or URL query parameters. Because the send mail script
checks for inputs in CGI inputs before importing from the
mailconfig
module,
it’s also possible to call this script outside the edit page to send
email—for instance, explicitly typing a URL of this nature into your
browser’s address field (but all on one line and with no intervening
spaces):

http://localhost:8000/cgi-bin/
onEditPageSend.py?site=smtp.rmi.net&
[email protected]&
[email protected]&
Subject=test+url&
text=Hello+Mark;this+is+Mark

will indeed send an email message as specified by the input
parameters at the end. That URL string is a lot to type into a browser’s
address field, of course, but it might be useful if generated
automatically by another script. As we saw in Chapters
13
and
15
, the module
urllib.request
can then be used to submit such a URL string to the server
from within a Python program.
Example 16-5
shows one way to
automate this.

"""
####################################################################
Send email by building a URL like this from inputs:
http://servername/pathname/
onEditPageSend.py?site=smtp.rmi.net&
[email protected]&
[email protected]&
Subject=test+url&
text=Hello+Mark;this+is+Mark
####################################################################
"""
from urllib.request import urlopen
from urllib.parse   import quote_plus
url = 'http://localhost:8000/cgi-bin/onEditPageSend.py'
url += '?site=%s'    % quote_plus(input('Site>'))
url += '&From=%s'    % quote_plus(input('From>'))
url += '&To=%s'      % quote_plus(input('To  >'))
url += '&Subject=%s' % quote_plus(input('Subj>'))
url += '&text=%s'    % quote_plus(input('text>'))    # or input loop
print('Reply html:')
print(urlopen(url).read().decode())   # confirmation or error page HTML

Running this script from the system command line is yet another
way to send an email message—this time, by contacting our CGI script on
a web server machine to do all the work. The script
sendurl.py
runs on any machine with Python and
sockets, lets us input mail parameters interactively, and invokes
another Python script that lives on a possibly remote machine. It prints
HTML returned by our CGI script:

C:\...\PP4E\Internet\Web\PyMailCgi>
sendurl.py
Site>
smtpout.secureserver.net
From>
[email protected]
To  >
[email protected]
Subj>
testing sendurl.py
text>
But sir, it's only wafer-thin...
Reply html:

PyMailCGI Confirmation

Send mail operation was successful
Press the link below to return to the main page.


align=left alt="[Python Logo]" border=0 hspace=15>
Back to root page

The HTML reply printed by this script would normally be rendered
into a new web page if caught by a browser. Such cryptic output might be
less than ideal, but you could easily search the reply string for its
components to determine the result (e.g., using the string
find
method or an
in
membership test to look for “successful”),
parse out its components with Python’s standard
html.parse
or
re
modules (covered in
Chapter 19
), and
so on. The resulting mail message—viewed, for variety, with
Chapter 14
’s
PyMailGUI program—shows up in this book’s email account as seen in
Figure 16-6
(it’s a single text-part
message).

Figure 16-6. sendurl.py result

Of course, there are other, less remote ways to send email from a
client machine. For instance, the Python
smtplib
module (used by
mailtools
) itself depends only upon the client
and SMTP server connections being operational, whereas this script also
depends on the web server machine and CGI script (requests go from
client to web server to CGI script to SMTP server). Because our CGI
script supports general URLs, though, it can do more than a
mailto:
HTML tag and can be invoked with
urllib.request
outside the context of
a running web browser. For instance, as discussed in
Chapter 15
, scripts like
sendurl.py
can be used to invoke and
test
server-side
programs.

If you flip back
to the main page in
Figure 16-2
, you’ll see a View link; pressing it
triggers the script in
Example 16-6
to run on the
server.

#!/usr/bin/python
"""
################################################################################
On view link click on main/root HTML page: make POP password input page;
this could almost be an HTML file because there are likely no input params yet,
but I wanted to use standard header/footer functions and display the site/user
names which must be fetched;  on submission, does not send the user along with
password here, and only ever sends both as URL params or hidden fields after the
password has been encrypted by a user-uploadable encryption module;
################################################################################
"""
# page template
pswdhtml = """


Please enter POP account password below, for user "%s" and site "%s".


Security note: The password you enter above will be transmitted
over the Internet to the server machine, but is not displayed, is never
transmitted in combination with a username unless it is encrypted or obfuscated,
and is never stored anywhere: not on the server (it is only passed along as hidden
fields in subsequent pages), and not on the client (no cookies are generated).
This is still not guaranteed to be totally safe; use your browser's back button
to back out of PyMailCgi at any time.

"""
# generate the password input page
import commonhtml                                         # usual parms case:
user, pswd, site = commonhtml.getstandardpopfields({})    # from module here,
commonhtml.pageheader(kind='POP password input')          # from html|url later
print(pswdhtml % (commonhtml.urlroot, user, site))
commonhtml.pagefooter()

This script is almost all embedded HTML: the triple-quoted
pswdhtml
string is printed, with
string formatting to insert values, in a single step. But because we
need to fetch the username and server name to display on the generated
page, this is coded as an executable script, not as a static HTML file.
The module
commonhtml
either loads
usernames and server names from script inputs (e.g., appended as query
parameters to the script’s URL) or imports them from the
mailconfig
file; either way, we don’t want to
hardcode them into this script or its HTML, so a simple HTML file won’t
do. Again, in the CGI world, we embed HTML code in Python code and fill
in its values this way (in server-side templating tools such as PSP the
effect is similar, but Python code is embedded in HTML code instead and
run to produce values).

Since this is a script, we can also use the
commonhtml
page header and footer routines to
render the generated reply page with a common look-and-feel, as shown in
Figure 16-7
.

Figure 16-7. PyMailCGI view password login page

At this page, the user is expected to enter the password for the
POP email account of the user and server displayed. Notice that the
actual password isn’t displayed; the input field’s HTML specifies
type=password
, which works just like
a normal text field, but shows typed input as stars. (See also the
pymail program in
Chapter 13
for doing
this at a console and PyMailGUI in
Chapter 14
for doing this in a tkinter
GUI.)

After you fill out the
last page’s password field and press its Submit button,
the password is shipped off to the script shown in
Example 16-7
.

#!/usr/bin/python
"""
################################################################################
On submit in POP password input window: make mail list view page;
in 2.0+ we only fetch mail headers here, and fetch 1 full message later upon
request; we still fetch all headers each time the index page is made: caching
Messages would require a server-side(?) database and session key, or other;
3.0: decode headers for list display, though printer and browser must handle;
################################################################################
"""
import cgi
import loadmail, commonhtml
from   externs import mailtools
from   secret  import encode       # user-defined encoder module
MaxHdr = 35                        # max length of email hdrs in list
# only pswd comes from page here, rest usually in module
formdata = cgi.FieldStorage()
mailuser, mailpswd, mailsite = commonhtml.getstandardpopfields(formdata)
parser = mailtools.MailParser()
try:
newmails = loadmail.loadmailhdrs(mailsite, mailuser, mailpswd)
mailnum  = 1
maillist = []                                           # or use enumerate()
for mail in newmails:                                   # list of hdr text
msginfo = []
hdrs = parser.parseHeaders(mail)                    # email.message.Message
addrhdrs = ('From', 'To', 'Cc', 'Bcc')              # decode names only
for key in ('Subject', 'From', 'Date'):
rawhdr = hdrs.get(key, '?')
if key not in addrhdrs:
dechdr = parser.decodeHeader(rawhdr)        # 3.0: decode for display
else:                                           # encoded on sends
dechdr = parser.decodeAddrHeader(rawhdr)    # email names only
msginfo.append(dechdr[:MaxHdr])
msginfo = ' | '.join(msginfo)
maillist.append((msginfo, commonhtml.urlroot + 'onViewListLink.py',
{'mnum': mailnum,
'user': mailuser,          # data params
'pswd': encode(mailpswd),  # pass in URL
'site': mailsite}))        # not inputs
mailnum += 1
commonhtml.listpage(maillist, 'mail selection list')
except:
commonhtml.errorpage('Error loading mail index')

This script’s main purpose is to generate a selection list page
for the user’s email account, using the password typed into the prior
page (or passed in a URL). As usual with encapsulation, most of the
details are hidden in other files:

The
maillist
list built here is
used to create the body of the next page—a clickable email message
selection list. Each generated hyperlink in the list page references a
constructed URL that contains enough information for the next script to
fetch and display a particular email message. As we learned in the
preceding chapter, this is a simple kind of state retention between
pages and scripts.

If all goes well, the mail selection list page HTML generated by
this script is rendered as in
Figure 16-8
. If your inbox is
as large as some of mine, you’ll probably need to scroll down to see the
end of this page. This page follows the common look-and-feel for all
PyMailCGI pages, thanks to
commonhtml
.

Figure 16-8. PyMailCGI view selection list page, top

If the script can’t access your email account (e.g., because you
typed the wrong password), its
try
statement handler instead produces a commonly formatted error page.
Figure 16-9
shows one that gives the
Python exception and details as part of the reply after a Python-raised
exception is caught; as usual, the exception details are fetched from
sys.exc_info
, and Python’s
traceback
module
is used to generate a
stack trace.

Figure 16-9. PyMailCGI login error page

The central
mechanism at work in
Example 16-7
is the generation of
URLs that embed message numbers and mail account information. Clicking
on any of the View links in the selection list triggers another script,
which uses information in the link’s URL parameters to fetch and display
the selected email. As mentioned in
Chapter 15
, because the list’s links are
programmed to “know” how to load a particular message, they effectively
remember what to do next.
Figure 16-10
shows part of the
HTML generated by this script (use your web browser View Source option
to see this for yourself—I did a Save As and then opened the result
which invoked Internet Explorer’s source viewer on my laptop).

Figure 16-10. PyMailCGI view list, generated HTML

Did you get all the details in
Figure 16-10
? You may not be
able to read generated HTML like this, but your browser can. For the
sake of readers afflicted with human-parsing limitations, here is what
one of those link lines looks like, reformatted with line breaks and
spaces to make it easier to understand:

View
Among our weapons are these | [email protected] | Fri, 07 May 2010 20:32...

PyMailCGI generates relative minimal URLs (server and pathname
values come from the prior page, unless set in
commonhtml
). Clicking on the word
View
in the hyperlink rendered from this HTML code
triggers the
onViewListLink
script as
usual, passing it all the parameters embedded at the end of the URL: the
POP username, the POP message number of the message associated with this
link, and the POP password and site information. These values will be
available in the object returned by
cgi
.
Field
Storage
in
the next script run. Note that the
mnum
POP message number parameter differs in
each link because each opens a different message when clicked and that
the text after

comes from
message headers extracted by the
mailtools
package, using the
email
package.

The
commonhtml
module escapes
all of the link parameters with the
urllib.parse
module, not
cgi.escape
, because they are part of a URL.
This can matter in the
pswd
password
parameter—its value might be encrypted and arbitrary bytes, but
urllib.parse
additionally escapes nonsafe characters in the encrypted
string per URL convention (it translates to
%xx
character sequences). It’s OK if the
encryptor yields odd—even nonprintable—characters because URL encoding
makes them legible for transmission. When the password reaches the next
script,
cgi.FieldStorage
undoes URL
escape sequences, leaving the encrypted password string without
%
escapes.

It’s instructive to see how
commonhtml
builds up the stateful link
parameters. Earlier, we learned how to use
the
urllib.parse.quote_plus
call to escape a
string for inclusion in URLs:

>>>
import urllib.parse
>>>
urllib.parse.quote_plus("There's bugger all down here on Earth")
'There%27s+bugger+all+down+here+on+Earth'

The module
commonhtml
, though,
calls the higher-level
urllib.parse.urlencode
function,
which translates a dictionary of
name:value
pairs into a complete URL query
parameter string, ready to add after a
?
marker in a URL. For instance, here is
url
encode
in action at the interactive
prompt:

>>>
parmdict = {'user': 'Brian',
...
'pswd': '#!/spam',
...
'text': 'Say no more, squire!'}
>>>
urllib.parse.urlencode(parmdict)
'text=Say+no+more%2C+squire%21&pswd=%23%21%2Fspam&user=Brian'
>>>
"%s?%s" % ("http://scriptname.py", urllib.parse.urlencode(parmdict))
'http://scriptname.py?text=Say+no+more%2C+squire%21&pswd=%23%21%2Fspam&user=Brian'

Internally,
urlencode
passes
each name and value in the dictionary to the built-in
str
function (to make sure they are strings),
and then runs each one through
urllib.parse.quote_plus
as they are added to
the result. The CGI script builds up a list of similar dictionaries and
passes it to
commonhtml
to be
formatted into a selection
list page.
^[
67
]

In broader terms, generating URLs with parameters like this is one
way to pass state information to the next script (along with cookies,
hidden form input fields, and server databases, discussed in
Chapter 15
). Without such state information,
users would have to reenter the username, password, and site name on
every page they visit along the way.

Incidentally, the list generated by this script is not radically
different in functionality from what we built in the PyMailGUI program
in
Chapter 14
, though the two differ
cosmetically.
Figure 16-11
shows this strictly client-side GUI’s view on the same email list
displayed in
Figure 16-8
.

Figure 16-11. PyMailGUI displaying the same view list

It’s important to keep in mind that PyMailGUI uses the tkinter GUI
library to build up a user interface instead of sending HTML to a
browser. It also runs entirely on the client and talks directly to email
servers, downloading mail from the POP server to the client machine over
sockets on demand. Because it retains memory for the duration of the
session, PyMailGUI can easily minimize mail server access. After the
initial header load, it needs to load only newly arrived email headers
on subsequent load requests. Moreover, it can update its email index
in-memory on deletions instead of reloading anew from the server, and it
has enough state to perform safe deletions of messages that check for
server inbox matches. PyMailGUI also remembers emails you’ve already
viewed—they need not be reloaded again while the program runs.

In contrast, PyMailCGI runs on the web server machine and simply
displays mail text on the client’s browser—mail is downloaded from the
POP server machine to the web server, where CGI scripts are run. Due to
the autonomous nature of CGI scripts,
PyMailCGI
by itself has no automatic
memory that spans pages and may need to reload headers and already
viewed messages during a single session. These architecture differences
have some important ramifications, which we’ll discuss later in this
chapter.