“Blessed are the poor in spirit: for theirs is the kingdom of heaven.”

Intelligent File Sharing Framework A THESIS IN Computer Science Changgyu Oh 5/2/2002

Contents Title Page Motivations3 Network Topologies4 Problem Domains5 Research Goal8 Related Works9 Intelligent File Sharing Framework12 Framework Figure13 Query Service Using Reasoning14 IS-A/Contained-In Hierarchies15 File Association Rules16 The Benefits of IFS Search17 Grouping Service18 IFS P2P V.S. P2P Network19 Benefits of Dynamic Group Partition20 Title Page Dynamic Group Partition 21 IP Clue Mechanism22 File Transaction in IFS24 QUERY SERVICE TYPES25 IFS System Architecture26 Client View27 Server View28 IFS Prototype Implementation29 IFS Query Interface30 Experimental Results32 Comparative Analysis33 Contributions34 Conclusion35 Future Work36 References37

Motivations Why P2P? –Limitations of Client/Server –Increasing interest in sharing and collaborative computing –Improving P2P technologies Why P2P File Sharing? –FILE Reusability –Share available resources Significance of this research –Increase Network scalability –Anonymity –Flexible and powerful query

Network Topologies

Problem Domains (1) Limitations of P2P Network –Scalability –Utilization of Network resources –P2P Network Topology Broadcast Logical Mesh network

Problem Domains (2) Limitation of Resource Source’s Anonymity –Resource source’s IP address in queryHit message Privacy and security How can source node send it to destination without revealing its IP address in public?

Problem Domains (3) Limitation of Keyword Based Query –Primitive and limited –Only one file searching –Not flexible –Not satisfy users’ requests

Research Goal To increase P2P network scalability Message flow control (Dynamic Group Partition and Caching) To protect the publisher anonymity IP-Clue mechanism (Encoding/Decoding) To increase the capacity of file querying File querying using intelligent reasoning, caching, dynamic peer group

Related Works-I Anonymous Publication Service The Publius system [Marc W., 2000] –document-anonymity because the key is split between the n servers, and without sufficient shares of the key a server is unable to decrypt the document that is stores. –Anonymity based on static, system-wide list of available servers. –Not support the adding of new server The Eternity system [Ross J., 1996] –Provides publisher’s anonymity by using one-way anonymous r ers –Server anonymity is not provided –Reader anonymity is not provided by open public proxies Query and Advertising System [Heimbigner D., 2000] –Arbitrary name is placed at the first level server for each client. –First level server has actual IP address of clients Freenet [Ian C., 2000] –Provides document-anonymity –Server-anonymity is not provided.

Related Works- II Meta Search Methods Efficient and Effective Metasearch [Yu C.,’1999] –representatives for each database optimizing relationship hierarchy Efficient Transitive Closure Reasoning [Lee Y.,2001] –inheritance, classification transitive closure reasoning –Class/Part/Containment Hierarchy Browsing Large Digital Library Collections [Geffner S., 1999] –classification hierarchies to increase capabilities of the data browsing in digital libraries.

Related Works-III File Sharing Systems using Caching The Distributed File System [Burns, R.C, 2000] –Detecting network failures ensures that caches are consistent. Network File System [Palmer J., 1996] –Clients poll the server to find out when the file was last modified –Determines the cached version is valid. Hint-Based Cooperative Caching file system [Sarkar, P., 2000] –Help clients make decisions based on the computer’s local state –Reduce overhead and access latency

Intelligent File Sharing Framework Major Building Blocks: –Query Service using Reasoning –IP-Clue Mechanism: Encoding/Decoding –Dynamic Grouping and Caching Service

Query Service Using Reasoning Goal: –Fast search using the file relation hierarchy Set –More flexible query and directory services Approach: –Relationships: IS-A Contained-In Run-With –File Relation Hierarchy Set Set of Number pairs (Ν), Relation Type (Ŗ), Constraint Rule (Ω), Hierarchy Identifier (Њ). –File Association Rules Generalized Association Rule Aggregated Association Rules Constrain-based Association Rule

IS-A/Contained-In Hierarchies

File Association Rules Generalized Association Rule –Subtype relationship between files –E.g., If Window multimedia application X is a multimedia application Y and if a multimedia file Z is running with the Multimedia application Y, then X runs Z. Aggregated Association Rule –directory contains multiple sub-directories or files –E.g., “Find the files on CS101 homework” Constrain-based Association Rule –File association based on constraints such as file size, Network capacity, etc. –E.g., “Find a file whose size is less than 1 MBtype and can be opened with MS Word.”

The Benefits of IFS Search MethodIFS SearchKeyword Based Search Keyword SearchYes File Extension SearchYesNo Application SearchYesNo Directory SearchYesNo Keyword Search in a certain directory YesNo File Extension Search in a certain directory YesNo File Search with ConstraintsYes CombinationYesNo

Grouping Service Goal: Increase Scalability –Control Maximum hop –Control a number of replicas of message generated by peer hosts –Control a number of peer hosts for message forwarding in a routing table of each peer host. Approach: –Group partition –Brother relationship –Caching

IFS P2P V.S. P2P Network

Benefits of Dynamic Group Partition Broadcast in a same group –Robust Search against node failure –Ensure a shortest path Increase Network Scalability by grouping peers –Server-less and Decentralized manner –Dynamic partition –Reduce network traffics Requires only one hop per a group

Dynamic Group Partition

IP Clue Mechanism Goal: Protect identity of resource publisher in P2P file sharing Approach –IP Encoding/Decoding Encoding the IP in source peers Decoding the encoded IP in destination peers –Formula: Assume that IP address of A is represented in [W.X.Y.Z] (e.g., [ ]) –(1) W + the size of query –(2) X + the first character of a query –(3) Y + the file extension size –(4) Z + the last character of a query message  Only the destination peers can recognize the IP Clue!!!

IP-Clue Mechanism

File Transaction in IFS

QUERY SERVICE TYPES

IFS System Architecture Component-based Architecture Servant Component –Highest level of component –Server + Client Components Manager Components: –Control work flow –Assign tasks to worker components Worker Components: –Perform actual tasks Service (Entity) Components: –Task description

Client View

Server View

IFS Prototype Implementation IFS prototype is built on top of Gnutella Phex System Developing System Environment –Need at least 25 Mbyte free Memory Space –JAVA Virtual Machine –Pentium III 500MHz CPU Event Driven Methods –Each task is performed based on events Components based Programming –Manager Components –Worker Components –Service Components

IFS Query Interface

Experimental Results Dynamic Group Partition and Cache

Comparative Analysis MeasureNapsterGnutellaIFS TopologyClient/ServerLogical Mesh Design Purpose MP3 file sharingFile sharing in a decentralized manner Enhanced Gnutella Size of Routing table Need a server’s IP address O(N)O(K) Where K << N Node Join Operation O(1) Node failureSevereTolerable Search Mechanism File indexing based on keyword search Fast Reasoning based on file association rules DescriptionClient/server based P2P network. Heavy traffics on servers Node failure is severe Decentralized Heave traffics due to the exponentially increased replicas of query messages Decentralized Control the network traffics Flexible query mechanism

Contributions Proposed a conceptual framework for decentralized P2P file sharing. –Dynamic group partition and caching –Query using fast reasoning –IP-clue mechanism (encoding/decoding) Designed a component-based architecture Implemented to extend an existing file sharing system (Gnutella Phex)

Conclusion The IFS system –Supports decentralized P2P File Sharing. –Increases high Network scalability. –Provides flexible file searching and querying. –Protect resource sources’ anonymity.

Future Work Further Research on the latency due to the grouping File registration strategy on heterogeneous environment Discover advanced mechanism to reasoning file relationships & file association rules Research on the grouping policies –Grouping by peer host’s network capacity –Grouping by interests –Grouping by context –Grouping by location

References: C. T. Yu, W. Meng, K.-L. Liu, W. Wu, and N. Rishe. Efficient and effective metasearch for a large number of text databases. In CIKM, pages , 1999 Y. Lee and J. Geller, Efficient Transitive Closure Reasoning in a Combined Class/Part/Containment Hierarchy, Journal of Knowledge and Information System, 2002 S. Geffner, D. Agrawal, A. Abbadi and T. Smith, Browsing Large Digital Library Collections Using Classification Hierarchies, CIKM, , 1999

References: (Continue) M. Waldman, A. Rubin, and L. F. Cranor. Publius: A robust, tamperevident, censorship-resistant, web publishing system. In Proc. 9th USENIX Security Symposium, page 59-72, August 2000 R. J. Anderson, The Eternity service, in Proceedings of the 1st International Conference on the Theory and Applications of Cryptology (PRAGOCRYPT '96), Prague, Czech Republic J. Palmer, R. Strong, and E. Upfal. Nonblocking membership protocols with asymmetric safety. Technical Report RJ10096 (91912), IBM Research Division, December 1997.

References: (Continue) I. Clarke, O. Sandberg, B. Wiley, and T. Hong. Freenet: A distributed anonymous information storage and retrieval system. In Proceedings of the Workshop on Design Issues in Anonymity and Unobservability, pages 46-66, July D. Heimbigner, Adapting Publish/Subscribe Middleware to Achieve Gnutella-like Functionality. Technical Report CU-CS , Department of Computer Science, University of Colorado, Sept P. Sarkar, J. H. Hartman ACM Transactions on Computer Systems (TOCS) November 2000 Volume 18 Issue 4