1. Proxy Cache Leonid Romanovsky Olga Fomenko Winter 2003 Instructor: Konstantin Sinyuk

2. Project Goals Part A: Find appropriate proxy software. Find appropriate proxy software. Integrate Proxy Cache Engine (PCE) card into proxy software. Integrate Proxy Cache Engine (PCE) card into proxy software. Initial testing. Initial testing. Part B: Bullet proof testing. Bullet proof testing. System and OS optimization. System and OS optimization.

3. System Overview Internet LAN Client Data Proxy Software

4. System Overview (Cont.) LAN Client Data Client Handler Functions Proxy cache engine (PCE) unit Proxy Handler Functions Search Request Search Results Page requested by client Internet

5. Project Steps Implementation of simple PCE simulator. Implementation of simple PCE simulator. Selection of proxy software Selection of proxy software Understanding Apache structure. Understanding Apache structure. Integrating PCE into Apache Integrating PCE into Apache Problem solving Problem solving Testing Testing System optimization System optimization OS optimization OS optimization

6. Selection of Proxy Software Apache with module proxy was chosen as proxy software due to its modularity Apache with module proxy was chosen as proxy software due to its modularity Apache 2.0.43 was chosen because it has separated modules for cache and proxy,in comparison with Apache 1.X. Apache 2.0.43 was chosen because it has separated modules for cache and proxy,in comparison with Apache 1.X.

7. Block Diagram Client Internet PCE card Apache 2.0.43 Web Interface mod_proxy Apache Request handler mod_disk_cache

8. Block Diagram(Cont.) Receive URL request Search in PCE database Fetch from cache Bring to cache Cache found Cache not found Update PCE database Handle request

9. Solved Problems Problem 1: PCE card drivers have no support in multithreaded and multiprocessing environments,as needed for Apache. PCE card drivers have no support in multithreaded and multiprocessing environments,as needed for Apache.

10. Solved Problems (Cont.) Solution 1: PCE card driver was integrated into web server interface with queue support. PCE card driver was integrated into web server interface with queue support. Into Apache was added web client interface. Into Apache was added web client interface. Host A with Proxy Host B with PCE card TCP/IP

11. Solved Problems (Cont.) Advantages of Solution 1: Future support for clustering and load- balancing. Future support for clustering and load- balancing. Better performance on heavy loaded servers. Better performance on heavy loaded servers. Disadvantages of Solution 1: Additional load on networks. Additional load on networks.

12. What’s next? Heavy loaded tests. Heavy loaded tests. System improvements and OS optimization. System improvements and OS optimization. (Maybe) Integrate into DigLab network environment. (Maybe) Integrate into DigLab network environment.

13. Ways to Improve Performance Rewrite code using Apache API Rewrite code using Apache API Add Win32 Messaging interface between PCE and Apache Add Win32 Messaging interface between PCE and Apache Build Windows XP Embedded or Windows PE as host OS for running Proxy Cache. Build Windows XP Embedded or Windows PE as host OS for running Proxy Cache.

14. Ways to Test Write web crawler. Write web crawler. Web crawlers start by parsing a specified web page, noting any hypertext links on that page that point to other web pages. They then parse those pages for new links, and so on, recursively. A crawler resides on a single machine. The crawler simply sends HTTP requests for documents to other machines on the Internet, just as a web browser does when the user clicks on links.

15. Time Table Build test environment20/10 – 28/10 Build test environment20/10 – 28/10 Testing and conclusions28/10 – 5/11 Testing and conclusions28/10 – 5/11 Optimization5/11 – 11/11 Optimization5/11 – 11/11 OS optimization11/11 – 16/11 OS optimization11/11 – 16/11 Final presentation16/11 – 30/11 Final presentation16/11 – 30/11