Presentation is loading. Please wait.

Presentation is loading. Please wait.

网络专题选讲 华中科技大学 电子与信息工程系 程文青 2013 年 1 月.

Similar presentations


Presentation on theme: "网络专题选讲 华中科技大学 电子与信息工程系 程文青 2013 年 1 月."— Presentation transcript:

1 网络专题选讲 华中科技大学 电子与信息工程系 程文青 2013 年 1 月

2 社会网络应用专题选讲 华中科技大学 电子与信息工程系 互联网技术与工程研究中心 黑晓军 Web:

3 《网络专题选讲 》 Outline Introduction Case study Traffic transport NetTube: Exploring Social Networks for Peer- to-Peer Short Video Sharing, 2009 Incentive P2P Trading in Social Networks: The Value of Staying Connected, 2010 Recommendation Circle-based Recommendation in Online Social Networks, 2012

4 Internet Topology

5 Introduction 我们生活在一个关系的社会 5

6 社会网络应用 6

7 Friend network in Facebook 7

8 Co-authorship network 8

9 Co-authorship in network science 9

10 Ingredient networks 10

11 911 事件 —— 犯罪网络 11

12 Social Networking ( General public )

13

14 Social Networking ( Academia )

15 专著

16

17 研究现状 主要研究机构 国外: MIT 、 Stanford 、 Maryland 、 USC 、 HP 、 Michigan 国内: IBM 中国研究院、微软亚洲研究院、中科院、中国传媒大学 、清华大学、南京大学 近年来社会网络成为国内外研究热点 美国国家科学基金会( NSF )将社会计算研究领域提供专项资金( 2010 ) 美国计算机协会 ACM , Workshop on Social Network Mining and Analysis(2007~2012) WWW 会议成立 “Social Network and Web2.0 Track” 论坛 (2009) SIGCOMM , ACM SIGCOMM Workshop on Online Social Networks(2009~2012) EuroSys, Workshop on Social Network System(2009~2012) 互联网测量会议( IMC ),海量数据仓库国际会议( VLDB ),信息与知识管理 ( CIKM )大量关于社交网络文章 全国网络科学论坛( 2004 ~ 2012 ),全国复杂网络会议( 2005 ~ 2011 ) 17

18 Exploring Social Networks for Peer-to-Peer Short Video Sharing NetTube: Exploring Social Networks for Peer-to-Peer Short Video Sharing Xu Cheng and Jiangchuan Liu School of Computing Science Simon Fraser University British Columbia, Canada October 2009 IEEE INFOCOM, 2009

19 Background (1) Social Networked Media Sharing – new killer Internet application Since 2005 Rich user-generated content (UGC) sharing Social networks Among users Among videos Changing the popular culture

20 Background (2) YouTube – a representative Popular Market share of around 43% More than six billion videos viewed in January 2009 Consumed as much bandwidth as the entire Internet in rd visits among all Internet sites (after Google and Yahoo) Fast growing 20% growth rate per month 15 hours of new videos are uploaded every minute

21 Motivation (1) The YouTube Crisis – all other sites’ challenge Severely hindered by client/server architecture Bandwidth costs Consumed as much bandwidth as the entire Internet in 2000 $1 million a day for server bandwidth! Sold to Google for $1.65 billion in Nov Performance and scalability “Slow” among the surveyed sites by Alexa.com

22 Motivation (2) Peer-to-peer (P2P) – alternative to Client/Server New generation of communication paradigm Each peer contributes its bandwidth to serve others Scale well with larger user base More users, more resources contributed Success already seen in BitTorrent, eMule, eDonkey (file sharing) Video broadcasting …

23 P2P 架构 没有永远在线的服务器 任意主机可以同另一个主 机进行通信 节点可以间歇性的连入系 统, IP 地址可能会变化 23 peer-peer

24 对等网络流媒体系统 两大设计空间 如何形成重叠网络? 如何传输内容? 现有体系结构 树状拓扑 + 推式内容传输 ESM, Yoid, CoopNet, SplitStream, Bullet, Chunkspread … 网状拓扑 + 拉式内容传输 -24- 《网络专题选讲》

25 网状 - 拉式对等网络流媒体系统 这类系统非常类似于 BitTorrent -25- 《网络专题选讲》

26 节点软件结构 双缓存 积极下载 vs 保守下载 处理丢失的数据块 缓存控制 -26- 《网络专题选讲》

27 缓存映像 (buffer map) 缓存映像反映了节点缓存所拥有的数据块信息 此映像可以被用来评估用户的播放质量 -27- 《网络专题选讲》

28 对等网络中的问题 内容组织和搜索 内容传输 信誉、激励及安全相关问题 28

29 CoolStreaming The first practical large-scale P2P NetTV Origin of data-driven mesh design With many follow-ups: PPLive, PPStream, UUSee … X. Zhang, J. Liu, B. Li, and T.-S. P. Yum, CoolStreaming/DONet: A Data- driven Overlay Network for Live Media Streaming, IEEE INFOCOM'05, March >800 citationsCoolStreaming/DONet: A Data- driven Overlay Network for Live Media Streaming J. Liu, S. G. Rao, B. Li, and H. Zhang, Opportunities and Challenges of Peer- to-Peer Internet Video Broadcast, Proceedings of the IEEE, Vol. 96, No. 1, pp , January 2008.Opportunities and Challenges of Peer- to-Peer Internet Video Broadcast

30 Data-Driven Mesh Core operations Every node periodically exchanges data availability information with a set of partners Then retrieves unavailable data from one or more partners, or supplies available data to partners Easy to implement no need to construct and maintain a complex global structure Efficient data forwarding is dynamically determined according to data availability Robust and resilient adaptive and quick switching among multi-suppliers

31 Challenges and Opportunities (1) Challenges – Drastically different statistics 1.5 year measurement of 5 million videos Short video clips – stability 99.6% are less than 700 seconds “I don’t want to wait for 30 seconds for a two-minute video!”  Searching for sources High churn rate: join/leave system Huge number of videos – scalability Highly skewed Inefficient for unpopular videos Very few users watch the same one

32 Challenges and Opportunities (2) Opportunities – Social networks No longer independent – videos have related videos Small-world – strong clustering Important role

33 NetTube Design (1) Bi-layer overlay network Lower-layer – per-video Download and uploading Peers stay in previous overlays as sources  Larger and more stable Upper-layer – social network Connected by the same peers in different lower-layer overlays Conceptual relation for searching Social network brings similar peers closer  Clustering  Efficient searching

34 NetTube Design (2) Bloom filter based indexing An efficient approach to keep track of peers’ cached videos Bloom filter An m-bit array using k hash functions Space-efficient Scalable indexing table Fast searching Table size is scalable with the number of videos Search locally and search in the upper-layer overlay  Social network clustering the similar video

35 NetTube Design (3) Transmission scheduling: From which partner to fetch which data segment ? Constraints Data availability Playback deadline Heterogeneous partner bandwidth Rarest-first (BitTorrent’s) doesn’t work !

36 NetTube Design (3) Variation of Parallel machine scheduling NP-hard Conventional Heuristics Message exchanged Window-based buffer map (BM): Data availability Segment request (piggyback by BM) Less suppliers first Multi-supplier: Highest bandwidth within deadline first

37 NetTube Design (3) Short video ? CODAS: Collaborative Delay-Aware Scheduling

38 NetTube Design (4) Social network assisted pre-fetching Most peers finish downloading before playback ends - free time available (about 80 seconds on average) Using free time to reduce startup delay Prefix pre-fetching  Avoid wasting bandwidth and space  Enable multiple pre-fetching Multiple pre-fetching  Accuracy increases greatly  Accuracy increases as watch more videos Pre-fetching among neighbors  Easy to implement  Social network helps improve efficiency

39 Performance Evaluation (1) Simulation Configuration Based on about 7,000 crawled videos Scale to more than 10,000 heterogeneous clients Compare with PA-VoD (MSN Video) Bandwidth reduction Save significantly more More scalable

40 Performance Evaluation (2) Simulation Impact of social network Find more sources: more than 95% within 2 hops Greatly increase pre-fetching accuracy

41 Performance Evaluation (3) PlanetLab experiment Configuration Maximum 235 PlanetLab nodes Experiment results Server bandwidth reduction: more than 40% Startup delay: average 2.2 s Playback continuity

42 Summary Contribution First social network assisted P2P system for short video sharing IWQoS’08, INFOCOM’09, IEEE Transactions on Multimedia Techniques Bi-layer overlay network Bloom filter based indexing Social network assisted pre-fetching Collaborative delay-aware scheduling Evaluation results Greatly reduce server bandwidth Much lower maintenance cost: $1 million → $60 K Inherently scalable – P2P Greatly reduce playback delay Satisfying startup delay Continuous playback

43 P2P Trading in Social Networks: The Value of Staying Connected Zhengye Liu, Hao Hu, Yong Liu, Keith Ross, Yao Wang, and Markus Mobius Polytechnic Institute of NYU Dept. of Economics, Harvard Unviversity 43 IEEE INFOCOM, 2010

44 Outline Background: P2P Incentive Networked Asynchronous Bilateral Trading (NABT) NABT Efficiency Theory NABT Simulations Conclusions 44

45 P2P Apps: BitTorrent 45

46 P2P Apps: Skype 46

47 Peer-Assisted Video Streaming Large scale deployments on Internet thousands of live/on-demand channels millions of world-wide users daily Leading P2P Video Companies CoolStreaming PPStream PPLive Sopcast UUSee

48 Major P2P Issues Traffic localization P4P Security Attacks on Attacks from Lack of uniform API Incentives for peers to contribute resources 48

49 Partially Successful P2P Incentive BitTorrent is popular 50+ client implementations Dozen public trackers 5-10 million users Why BitTorrent? First generation P2P applications: Gnutella 70% of users are free-riders Second generation P2P applications: BitTorrent P2P design Resources Incentives +  49

50 The BitTorrent Incentive Implementation of incentive: The rich play/trade with the rich To get files faster… contribute more bandwidth 50

51 BitTorrent: Tit-for-tat (1) Alice tries sending to Bob. Is he rich? (2) Alice becomes one of Bob’s top-four providers; Bob reciprocates. (3) Bob becomes one of Alice’s top-four providers. (0) Everyone nominally has four trading partners

52 Tit-for-Tat: Live P2P Video “LayerP2P: Using Layered Video Chunks in P2P Live Streaming”, Z. Liu, Y. Shen, K.W. Ross, S. Panwar, Y. Wang, IEEE Transactions on Multimedia, November “LayerP2P: Using Layered Video Chunks in P2P Live Streaming”, Z. Liu, Y. Shen, K.W. Ross, S. Panwar, Y. Wang, IEEE Transactions on Multimedia, November To get better video quality… contribute more bandwidth “Substream Trading: Towards an Open P2P Live Streaming System”, Z. Liu, Y. Shen, K.W. Ross, S. Panwar, Y. Wang, Inter Conf on Network Protocols (ICNP), October 2008 “Substream Trading: Towards an Open P2P Live Streaming System”, Z. Liu, Y. Shen, K.W. Ross, S. Panwar, Y. Wang, Inter Conf on Network Protocols (ICNP), October

53 Limitations of Tit-for-Tat Tit-for-Tat is synchronous trading Alice and Bob can trade if and only if they simultaneously have data for each other in a short time period Tit-for-Tat == Barter ( 物物交换 ) in primitive economy Barter is highly inefficient fails if lack of “double coincidence of wants” failure example: Tit-for-tat does not provide incentive for seeding 53

54 Currency-based Trading Currency improves trading efficiency in modern economy Asynchronous trading regulated by money users accumulate for providing services and later spend for acquiring services 54

55 Major Issues/Solutions  Cheating  Counterfeit  Dispute Resolution Solutions:  Banking System  Market Regulation  Trading Policy  Court System  Law Enforcement  …… 55

56 Global Currency in P2P? Peers trade with each other using digital cash earn cash by contributing resources to provide services to other peers, pay cash to consume services provided by other peer. Heavyweight coordination infrastructure needed banking/regulation/court/enforcement hard to justify for P2P trading goods carrying low value. Only limited research attempts, no large-scale deployment 56

57 Desirable P2P Incentive Mechanism High Trading Efficiency trade asynchronously trade with many peers trade diverse set of goods/services Cheating-proof isolate and punish cheaters prohibit collusions Low-degree of Coordination light-weight and distributed protocols low management cost 57

58 Outline Background: P2P Incentive Networked Asynchronous Bilateral Trading (NABT) NABT Efficiency Theory NABT Simulations Conclusions 58

59 Alternative Trading Systems in Social Networks Asynchronous Trading exploit trust between friends allow debt: providing a service without immediate payment Networked Trading exploit trust in network of friends trade with indirect friends “Trust and social collateral”. Dean Karlan, Markus Mobius, Tanya Rosenblat, and Adam Szeidl. Quarterly Journal of Economics, “Trust and social collateral”. Dean Karlan, Markus Mobius, Tanya Rosenblat, and Adam Szeidl. Quarterly Journal of Economics,

60 Friendship as Trading Collateral ( 抵押 ) Resolve cheating/disputes: Terminate friendship! 60

61 P2P Trading in Social Networks Networked Asynchronous Bilateral Trading (NABT) Social network: peers belong to an underlying social network Pair-wise credit: friends maintain pair-wise credits Asynchronous trading: peers can use their credits anytime they want Credit limit: each peer sets a credit limit for each of its friends Networked trading: peer trades with a remote peer by transferring credits through a chain of friends links. 61

62 Async Trading Between Direct Friends A pair of friends maintain local credit balance b ij = amount of credits that j owes i b ij =-b ji update balance upon services Control risk of defaulting C ij = credit limit for j set by i - C ji ≤ b ij ≤ C ij incentivizes users b AB b BA Alice Bob + ∆ - ∆ 62

63 Networked Trading via Intermediaries To access service on a remote peer 1. find a path of friend links in social network 2. arrange a series of credit transfers along path 3. intermediaries update credit balances with upstream and downstream friends, and break even 4. remote peer provides requested service b AB + ∆ Alice Bob, b BA -∆, b BC + ∆ b CB - ∆ Charlie 63

64 NABT Issues NABT is decentralized, and effective for resolving disputes. But Is NABT efficient? How to set credit limits C ij ? Can users free-ride in NABT? 64

65 Outline Background: P2P Incentive Networked Asynchronous Bilateral Trading (NABT) NABT Efficiency Theory NABT Simulations Conclusions 65

66 NABT Efficiency Single trade can be exercised if and only if a credit transfer can be arranged subject to social network connectivity obey credit limit on each social link Multiple trades coupled through the underlying social network later trades work with credit balance resulted from earlier trades concurrent trades compete for credit transfer 66

67 NABT Efficiency Model Given: underlying social network: credit limits as link weights: service demand matrix: : cost charged by user k to serve user l. Find credit transfer flows for all demands : credit flow for demand d on social link credit flow conservation on intermediaries resulting credit balance bounded by credit limits 67

68 NABT Credit Flow Routing Similar to classical network flow problem, but: credit balance on link can be negative credit flows in opposite directions cancel Example: Circular Service Demands: A wants a file on B, B wants a file on C, and C wants a file on A B A C credit routing scheme 1 b BA =1 b CB =1 b AC =1 B A C credit routing scheme 2 b BA =0 b CB =0 b AC =0 68

69 Balanced Demand For each user k, total service he provides (regardless of receivers) equals total service received (regardless of providers) Theorem 1: Any balanced demand can be executed as long as users involved in the demand sets are connected. NABT is as efficient as global currency networked Tit-for-Tat: peers play tit-for-tat with whole network instead of another peer 69

70 Unbalanced Demand For at least one user, service contribution does not equal to service consumption. net-service contribution: service sources: service sinks: aggregate net-service imbalance 70

71 Extended Social Network augment social network with a virtual source, a virtual sink, virtual links Example aggregate net-service imbalance 71

72 Efficiency with Unbalanced Demand Theorem: An unbalanced demand is executable iff the min-cut between the source s+ and sink s- in extended social network is greater than or equal to the aggregate net-service imbalance. What matters: underlying social network topology credit limits on social links service imbalance between a user and whole network What does not matter: service imbalance between individual pairs of users 72

73 Dynamic Payment Routing Time is slotted Demands are now sequential H(1), H(2),… Suppose we succeed at executing H(1),…, H(k-1). Theorem: To successfully execute H(k), we do not have to worry about how we executed H(1),…,H(k-1). 73

74 Outline Background: P2P Incentive Networked Asynchronous Bilateral Trading (NABT) NABT Efficiency Theory NABT Simulations Conclusions 74

75 Preliminary NABT Protocol Design On-demand credit flow routing locate service providers send out credit-transfer request through controlled flooding request propagates along friends links with enough credit space When request hits one providers, it sends back reply through reverse path to establish credit transfer on intermediaries. Complete credit transfer and service Dynamic credit-limit setting increase credit-limit linearly after each fulfilled transaction decrease credit-limit multiplicatively after each unfulfilled/disputed transaction 75

76 Simulation Study Trading with global currency (GCT): Global currency and a centralized bank Each peer has Bi initial credits and each file costs one credit If peer i downloads a file from peer j, peer i pays 1 credit to peer j Synchronous Trading (ST): Two peers can trade if and only if they can supply files to each other simultaneously If peer i downloads a file from peer j, peer j will download a file from peer i. Two-hop NABT: Peers are connected in an underlying social network A requesting peer requests files from its friends (one-hop friends) and the friends of its friends (two-hop friends) If peer i downloads a file from peer j within two hops, peer i passes 1 credit to peer j 76

77 Simulation Setup Peer profile Social network with a topology collected from MySpace Totally 10,000 peers Peer upload bandwidth 37% Ethernet users (1.2Mbps) + 63% residential users (400 kbps) Willingness for sharing 10% content-rich peers (1,000 files) + 90% content-scarce peers (50 files) Online and offline Markov ON-OFF process (On time = Off time = 12 hours) File profile Totally 10,000 different files Files are small and have the same size of 3MB File popularity follows a Zipf distribution 77

78 Trading Efficiency Request success ratio: The ratio of fulfilled requests to the total number of requests CDF of request success ratio 78

79 Importance of Trading Intermediaries CDF of request success ratio for the systems with and without intermediaries 79

80 Service Differentiation of NABT Relation between request success ratio and upload contribution (in terms of number of uploaded files) 80

81 Conclusion NABT -- a new P2P trading paradigm over social networks exploits trust between friends, and friends network trade asynchronously, and over network, light-weight, distributed NABT is efficient almost as efficient as global currencies support networked tit-for-tat topology and credit limits matters memoryless 81

82 Open Research Issues incentives for intermediaries isolate and punish cheators dynamic credit-limit setting heterogeneous NABT market diverse set of services exchange ratio between pair-wise credits deal-making … … 82

83 Take Away Messages Asynchronous incentives are critical for taking P2P to the next level Async incentives require money The future of P2P may lie in social networks 83

84 84 Circle-based Recommendation in Online Social Networks Xiwang Yang, Harald Steck*, and Yong Liu Polytechnic Institute of NYU * Bell Labs/Netflix 84 ACM KDD 2012

85 85 Outline Background & Motivation Circle-based RS Trust Circle Inference Trust Value Assignment Model Training Evaluation Conclusion & Future work 85

86 Social Recommenders Everywhere 86

87 Collaborative Filtering (CF) Most Used and Well Known Approach for Recommendation Finds Users with Similar Interests to the target User Aggregating their opinions to make a recommendation. 87

88 User Based Collaborative Filtering TargetCustomer Aggregator Prediction 88

89 Item-based Collaborative Filtering 89

90 Item-Item Collaborative Filtering Aggregator Prediction 90

91 Matrix Factorization (BaseMF) [NIPS08] 91  Introduced by R. Salakhutdinov and A. Mnih Probabilistic matrix factorization. In NIPS 2008  Model based approach  Latent features for users  Latent features for items P and Q have normal priors

92 Matrix Factorization (BaseMF) 92  Prediction Model  Objective Function P and Q have normal priors

93 Related Work-Social Recommender Social Recommendation (SoRec) Model CIKM’08 Factorizing social trust matrix together with user rating matrix Social Trust Ensemble (STE) Model SIGIR’09 User’s rating influenced by social friends SocialMF Model RecSys’10 User’s latent feature (taste) influenced by social friends Handle trust propagation in social network Using whole trust network for item rating prediction 93

94 SocialMF [RecSys2010] Social Influence  behavior of a user u is affected by his direct neighbors. Latent factor of a user depend on his neighbors. is the normalized trust value. Prediction Model: Objective: 94

95 Proposed Improvements for Current Social Recommender Social networks include multiple circles A more refined social trust information—richer information Incorporate circle information in Social Recommender Use trust circles specific to an item category when predict rating in this category e.g. Trust Circle of “Music”, Trust Circle of “Cars”, etc 95

96 Proposed Improvements for Current Social Recommender 96 Existing circles (Google+, Facebook) not corresponding to an item category

97 Proposed Improvements for Current Social Recommender In existing multi-category rating datasets, no circle information User trusts different subsets of friends in different domains (Cars, Music…) User trusts different friends differently, related to friend’s expertise value Should use trust circle specific to item category 97

98 98 Outline Background & Motivation Circle-based RS Trust Circle Inference Trust Value Assignment Model Training Evaluation Conclusion & Future work 98

99 Trust Circle Inference User v is in inferred circle c of u iff u trust v in original social network and both of them have rating in category c 99 Original Social Network Inferred circle for category C1 Inferred circle for category C 2 Inferred circle for category C 3

100 100 Outline Background & Motivation Circle-based RS Trust Circle Inference Trust Value Assignment Model Training Evaluation Conclusion & Future work 100

101 Trust Value Assignment CircleCon1: Equal Trust 101

102 Trust Value Assignment CircleCon2: Expertise-based Trust assign a higher trust value or weight to the friends that are experts in the circle / category. 102

103 CircleCon2: Expertise-based Trust Variant a: Expertise based on number of ratings in a circle 103

104 CircleCon2: Expertise-based Trust Variant b: 104 D w records the proportions of ratings user w assigned in all categories. It reflects the interest distribution of w cross all categories

105 CircleCon3: Trust Splitting Trust due to followee’s rating in one category Likelihood u2 trusts u1 in C1, C2 ? Infer likelihood proportional on u2’s number of ratings in C1 and C2. Assign trust value in a category proportional to the likelihood u2 trusts u1 in a category 105 Original trust link trust link in c 1 trust link in c 2

106 CircleCon3: Trust Splitting Normalize across followees 106

107 107 Outline Background & Motivation Circle-based RS Trust Circle Inference Trust Value Assignment Model Training Evaluation Conclusion & Future work 107

108 Model Training Training with ratings from each category Predict user’s rating in category c Input rating: rating in category c Input social network: Circle c 108   is the number of items in category c Solved by gradient descent  is social information weight

109 Model Training Training with ratings from each category 109

110 Model Training Training with ratings for all categories Predict user’s rating in category c Input rating: rating from all categories Input social network: Circle c 110

111 111 Outline Background & Motivation Circle-based RS Trust Circle Inference Trust Value Assignment Model Training Evaluation Conclusion & Future work 111

112 Epinions Data 112

113 Performance Metrics 113

114 Training with per-category ratings 114

115 Training with per-category ratings 115

116 Training with ratings from all categories 116 CircleCon3 of training with per-category rating

117 Training with ratings from all categories 117

118 Training with ratings from all categories 118

119 Summary Propose a novel Circle-based Social Recommendation framework Split original social network to different circles, one circle corresponding to one item category User trusts different subsets of friends in different domains(Cars, Music…) User trusts different friends differently, based on friend’s expertise Outperforms the state-of-the-art social collaborative filtering algorithms Show the promising future of circle-construction techniques in Social Recommender 119

120 小结 120 Social networking has been changing the way which people communicate!

121 Reading List Lada Adamic, Social Network Analysis, https://class.coursera.org/sna /wiki/view?page=syllabushttps://class.coursera.org/sna /wiki/view?page=syllabus World By David Easley and Jon Kleinberg, Networks, Crowds, and Markets Reasoning About a Highly Connected, Cambridge University Press, Xu Cheng and Jiangchuan Liu, "NetTube: Exploring Social Networks for Peer-to-Peer Short Video Sharing", IEEE INFOCOM, Zhengye Liu, Hao Hu, Yong Liu, Keith Ross, Yao Wang, and Markus Mobius, “P2P Trading in Social Networks: The Value of Staying Connected”, in the Proceedings of IEEE Conference on Computer and Communications IEEE INFOCOM, 2010 Xiwang Yang, Harald Steck and Yong Liu, “Circle-based Recommendation in Online Social Networks ”, in the Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2012), Long Paper, August,


Download ppt "网络专题选讲 华中科技大学 电子与信息工程系 程文青 2013 年 1 月."

Similar presentations


Ads by Google