Presentation is loading. Please wait.

Presentation is loading. Please wait.

Presenter: Christian Charette, John Tkaczewski Duration: 30 minutes May Webinar: 10Gbps transfers.

Similar presentations


Presentation on theme: "Presenter: Christian Charette, John Tkaczewski Duration: 30 minutes May Webinar: 10Gbps transfers."— Presentation transcript:

1 Presenter: Christian Charette, John Tkaczewski Duration: 30 minutes May Webinar: 10Gbps transfers

2 Presenter Introduction | © Copyright FileCatalyst, 2013 2 John Tkaczewski Co-Founder and President of Unlimi-Tech Software Oversees creative and technical vision for FileCatalyst Webmail and FileCatalyst Workflow Blog: http://www.filecatalyst.com/bloghttp://www.filecatalyst.com/blog Christian Charette Director of Software Development Project lead on 10 Gbps research

3 Agenda FileCatalyst Direct Overview Quick refresher why TCP doesn’t work in high speed networks 10Gbps test infrastructure – Ie: what are we using to perform the demo? Watch the data fly! Questions (please type them in as we go) | © Copyright FileCatalyst, 2013 3

4 FileCatalyst Direct Overview | © Copyright FileCatalyst, 2013 4 Software architecture is Client-Server. One side acts as the main file repository (FileCatalyst Server) One or many clients connect to server, and push/pull data Clients include: – automated interfaces (API, CLI, Javascript, C++) – web components (applets) – feature rich applications (HotFolder, Express, mobile) Primarily written in Java, works on every major platform (Windows, Linux, Mac OSX, iOS, Android, and more) Transport protocol utilizes proprietary UDP, with error correction and congestion control built into the application layer (instead of relying on TCP network layer)

5 Putting 10 Gbps in context | © Copyright FileCatalyst, 2013 5 10 Gbps transfers an average BlueRay disk (26GB) in under 30 seconds Cost of 10 Gbps dedicated lines is dropping – becoming cost effective to many companies (~10K/month US to UK) 1 Gbps fiber connections becoming available to residential consumers

6 TCP vs UDP for packet loss & latency | © Copyright FileCatalyst, 2013 6 Quick refresher: TCP can’t cut it. Data provided as part of Photonics North 2012 presentation, FTP data uses FileZilla client connecting to PureFTPDPhotonics North 2012

7 TCP vs UDP for packet loss & latency | © Copyright FileCatalyst, 2013 7 TCP cannot handle packet loss -- even for networks with small latency. High latency networks fall apart under real world conditions.

8 TCP vs UDP for packet loss & latency | © Copyright FileCatalyst, 2013 8 You cannot transfer files at 10 Gbps with TCP. It just doesn’t work.

9 10 Gbps transfers | © Copyright FileCatalyst, 2013 9 10 Gbps research project started in May 2011. Project completed 6 months later with Nov 2011 release of FileCatalyst Direct v3.0, able to transfer at full 10gbps speed. 1½ years of successful deployments of 2-10 Gbps in the field. Tested in 2012 by independent researchers (ex-CEO of competitor Tixel GmbH), validated as most stable transfer results between 5 managed transport solutions. Comparison of Contemporary Solutions for High Speed Data Transport on WAN 10 Gbit/s ConnectionsComparison of Contemporary Solutions for High Speed Data Transport on WAN 10 Gbit/s Connections, Dmitry Kachan, Eduard Siemens, Vyacheslav Shuvalov

10 What are we using to run this demo? | © Copyright FileCatalyst, 2013 10 FileCatalyst HotFolder Ares x.x.x.58 FileCatalyst Server Hermes x.x.x.59

11 What are we using to run these tests? | © Copyright FileCatalyst, 2013 11 Machines utilize standard components circa 2011. Each machine costs approx. $5000 to put together. – Intel Westmere 5600 series (used in demo testbed) – AMD Magny-Cours 6100 Opterons FileCatalyst HotFolder Ares x.x.x.58 Dual Intel Xeon 5690 48 GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu 10.04.1 LTS Java 1.7 64-bit FileCatalyst v3.2 FileCatalyst Server Hermes x.x.x.59 Dual Intel Xeon 5690 48 GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu 10.04.1 LTS Java 1.7 64-bit FileCatalyst v3.2

12 What are we using to run these tests? | © Copyright FileCatalyst, 2013 12 Machine has 48GB total, 36GB reserved for RAMFS (so 12GB for OS + FileCatalyst). For the purpose of these tests, going to read/write to RAM drive. Why? Want to remove disks as part of the equation for demo to simplify things (everybody has different IO solutions in place). If your disks can handle the speeds, our software can transfer it. FileCatalyst HotFolder Ares x.x.x.58 Dual Intel Xeon 5690 48 GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu 10.04.1 LTS Java 1.7 64-bit FileCatalyst v3.2 FileCatalyst Server Hermes x.x.x.59 Dual Intel Xeon 5690 48 GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu 10.04.1 LTS Java 1.7 64-bit FileCatalyst v3.2

13 What are we using to run these tests? | © Copyright FileCatalyst, 2013 13 In demo, to simulate WAN environment, a Linktropy box from Apposite Technologies is used – Control link speed, packet loss, and link latency FileCatalyst HotFolder Ares x.x.x.58 Dual Intel Xeon 5690 48 GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu 10.04.1 LTS Java 1.7 64-bit FileCatalyst v3.2 FileCatalyst Server Hermes x.x.x.59 Dual Intel Xeon 5690 48 GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu 10.04.1 LTS Java 1.7 64-bit FileCatalyst v3.2 WAN Emulator Apposite Technologies

14 So what are we going to see? | © Copyright FileCatalyst, 2013 14 FileCatalyst HotFolder Ares x.x.x.58 Dual Intel Xeon 5690 48 GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu 10.04.1 LTS Java 1.7 64-bit FileCatalyst v3.2 FileCatalyst Server Hermes x.x.x.59 Dual Intel Xeon 5690 48 GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu 10.04.1 LTS Java 1.7 64-bit FileCatalyst v3.2 WAN Emulator Apposite Technologies

15 So what are we going to see? | © Copyright FileCatalyst, 2013 15 (1) Going to test most basic scenario: – Transfer static file from point A to B – 30GB file… slightly larger than the average Blue Ray disk. FileCatalyst HotFolder Ares x.x.x.58 Dual Intel Xeon 5690 48 GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu 10.04.1 LTS Java 1.7 64-bit FileCatalyst v3.2 FileCatalyst Server Hermes x.x.x.59 Dual Intel Xeon 5690 48 GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu 10.04.1 LTS Java 1.7 64-bit FileCatalyst v3.2 WAN Emulator Apposite Technologies

16 So what are we going to see? | © Copyright FileCatalyst, 2013 16 (2) Going to test standard live media scenario: – Growth files (5) simulate live video feeds saved to system – Move files in real-time to destination as files are output FileCatalyst HotFolder Ares x.x.x.58 Dual Intel Xeon 5690 48 GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu 10.04.1 LTS Java 1.7 64-bit FileCatalyst v3.2 FileCatalyst Server Hermes x.x.x.59 Dual Intel Xeon 5690 48 GB of RAM (36 GB as RAMDRIVE ) Intel 10Gbps cards Ubuntu 10.04.1 LTS Java 1.7 64-bit FileCatalyst v3.2 WAN Emulator Apposite Technologies

17 What are we using to run these tests? | © Copyright FileCatalyst, 2013 17 Configurations used for single file transfer @ 10gbps: – Server & HotFolder each using 4GB of RAM – MTU 9000 (packet 8972 bytes) – Blocks: 10MB – Number of sender threads: 50 – Sender sockets: 5 – Receiver sockets: 3 – RTT congestion control (because test scenario utilizes packet loss) – Congestion control set to 7 (more aggressive) Linktropy setup to simulate LA -> NY transfers – Latency of 60 ms – 0.1% packet loss

18 DEMOS | © Copyright FileCatalyst, 2013 18

19 Upcoming Events: - NYC Digital Asset Managers, (May 29 th ) - Broadcast Asia, Singapore (June 18-21) - IBC, Amsterdam (Sept 13-17) - Interop New York (Oct 2-3) - Broadcast India, Mumbai (Oct 9-11) And of course don’t forget our monthly webinar....

20 Thank you for attending our May webinar! Questions? Please type in your questions.


Download ppt "Presenter: Christian Charette, John Tkaczewski Duration: 30 minutes May Webinar: 10Gbps transfers."

Similar presentations


Ads by Google