1. Retrieving Web Pages (HTTP), Topic 3, Chapter 6
Network Programming
Kansas State University at Salina

2. First, some comments Switch to application protocols Client side focus
Pre-build Modules
A natural OO thing – a matter of productivity
Argh!, someone else’s code
Lots of choices, language independent principles
Web related network programming
Chapter 6 – retrieving web pages – easy
Chapter 7 – Parsing HTML – hard
Chapter 8 – XML and XML-RPC – interesting

3. HTTP Basics Stateless, connectionless protocol Basic GET …
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((' 80))
request = """GET /faculty/tim/index.html HTTP/1.0\n
From:
User-Agent: Python\n
\n"""
s.send(request)
fp = open( "index.html", "w" )
while 1:
data = s.recv(1024)
if not len(data):
break
fp.write(data)
s.close()
fp.close()

4. Now, for the easy way … import sys, urllib2
page = "
req = urllib2.Request(page)
fd = urllib2.urlopen(req)
while 1:
data = fd.read(1024)
if not len(data):
break
sys.stdout.write(data)

5. Submitting with GET >>> import urllib
>>> encoding = urllib.urlencode( [('activity', 'water ski'), \
('lake', 'Milford'), ('code', 52)] )
>>> print encoding
activity=water+ski&lake=Milford&code=52
>>> url = " + '?' + encoding
>>> print url

6. Submitting with POST
>>> encoding = urllib.urlencode( [('activity', 'water ski'),\
('lake', 'Milford'), ('code', 52)] )
>>> print encoding
activity=water+ski&lake=Milford&code=52
>>> import urllib2
>>> req = urllib2.Request(url)
>>> fd = urllib2.urlopen(" encoding)