Portable binary serialization, the Google way

Slides:



Advertisements
Similar presentations
.Net Remoting by James Stone. What is.Net Remoting? RemotingRemoting is a technology that allows.NET applications to communicate. Remoting Same machine,
Advertisements

General introduction to Web services and an implementation example
SATO XML-enabled Printers Introduction
De-mystifying Google’s hottest binary protocol Prasanna Kanagasabai Jovin Lobo.
SOAP.
CIS 375—Web App Dev II SOAP.
XML in the real world (2) SOAP. What is SOAP? ► SOAP stands for Simple Object Access Protocol ► SOAP is a communication protocol ► SOAP is for communication.
Web Services Darshan R. Kapadia Gregor von Laszewski 1http://grid.rit.edu.
CSCI-1680 RPC and Data Representation Rodrigo Fonseca.
Streaming NetCDF John Caron July What does NetCDF do for you? Data Storage: machine-, OS-, compiler-independent Standard API (Application Programming.
May 2, 2006Shawn Mulkey - EECS Distributed Computing & Object Oriented Middleware: Part 2 Presented By Shawn Mulkey.
.NET Remoting in Delphi and C# Alain “Lino” Tadros President & CEO Falafel Software Inc. ComponentScience Inc. BORCON 2004.
C# and Windows Programming Application Domains and Remoting.
GridRPC Sources / Credits: IRISA/IFSIC IRISA/INRIA Thierry Priol et. al papers.
.NET Remoting. .Net Remoting Replaces DCOM (Distributed Component Object Model – a proprietary Microsoft technology for communication among software components.
Introduction to Web Services
Windows Communication Foundation and Web Services.
J2ME Web Services Specification.  With the promise to ease interoperability and allow for large scale software collaboration over the Internet by offering.
Distributed Systems CS Google Protocol Buffers and Publish-Subscribe Recitation 3, Sep 22, 2011 Majd F. Sakr, Vinay Kolar, Mohammad Hammoud.
MIT iCampus iLabs Software Architecture Workshop June , 2006.
The Microsoft Technical Roadshow 2006 Windows Communication Foundation Mike Taulty Developer & Platform Group Microsoft Ltd
Client-server interactions in Mobile Applications.
Jon Flanders INT303. About Me  Jon Flanders –  Independent consultant/trainer  BizTalk MVP.
Data Formats CMSC 491 Hadoop-Based Distributed Computing Spring 2015 Adam Shook.
CSCI-1680 RPC and Data Representation Rodrigo Fonseca.
Avro Apache Course: Distributed class Student ID: AM Name: Azzaya Galbazar
It’s always better live. MSDN Events INTRODUCTION TO SILVERLIGHT prepared by Joe Nov INTRODUCTION TO SILVERLIGHT prepared by Joe Nov
Agenda What Is the Windows Communication Foundation? How Does It Work? How Do I Use and Deploy It? Bindings Addresses Contracts How to host WCF services.
XML Overview. Chapter 8 © 2011 Pearson Education 2 Extensible Markup Language (XML) A text-based markup language (like HTML) A text-based markup language.
Chapter 17 - Deploying Java Applications on the Web1 Chapter 17 Deploying Java Applications on the Web.
Serialization  What is Serialization?  System.Serialization  Scenarios in Serialization  Basic Serialization  Custom Serialization.
Microsoft Visual Studio 2010 Muhammad Zubair MS (FAST-NU) Experience: 5+ Years Contact:- Cell#:
Big Data Open Source Software and Projects ABDS in Summary I: Layers 1 to 2 Data Science Curriculum March Geoffrey Fox
WEB BASED DATA TRANSFORMATION USING XML, JAVA Group members: Darius Balarashti & Matt Smith.
Web Services. ASP.NET Web Services  Goals of ASP.NET Web services:  To enable cross-platform, cross- business computing  Great for “service” based.
Serialization What is Serialization Serialization is the process of converting an object, or a connected graph of objects, stored within computer memory,
Mahesh Krishnan, Senior Consultant, Readify Slide 1.
Telerik Software Academy Web Services & Cloud.
Simple Object Access Protocol. Web Services: SOAP2 Why Simple Object Access Protocol Light weight replacement for complicated distributed object technology.
Copyright © 2012 UNICOM Systems, Inc. Confidential Information z/Ware Product Overview illustro Systems International A Division of UNICOM Global.
COS 461 Recitation 7 Remote Procedure Calls. Let’s Look at Layers Again.
S imple O bject A ccess P rotocol Karthikeyan Chandrasekaran & Nandakumar Padmanabhan.
S O A P ‘the protocol formerly known as Simple Object Access Protocol’ Team Pluto Bonnie, Brandon, George, Hojun.
What is SOAP? Luciano Piccoli. SOAP – Simple Object Access Protocol  SOAP is an XML based protocol to let software components and applications communicate.
ASP.NET Web Services.  A unit of managed code installed under IIS that can be remotely invoked using HTTP.
Mock Objects in Functional Testing Sven Rosvall. Dimension Data Cloud Business Unit.
© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/3 JSP Application Models.
Web Services from 10,000 feet Part I Tom Perkins NTPCUG CertSIG XML Web Services.
Presentation 24: Windows Communication Foundation Introduced Objektorienteret Netværkskommunikation.
.NET Mobile Application Development XML Web Services.
.NET XML Web Services by Joe Mayo Mayo Software Consulting
Integrating and Extending Workflow 8 AA301 Carl Sykes Ed Heaney.
OE-NIK HP Advanced Programming WCF WCF SOAP service, host, client Exception handling (NOT required)
OE-NIK HP Advanced Programming Web services Standards and protocols Using web services Using web services with asynchronous calls.
Apache Avro CMSC 491 Hadoop-Based Distributed Computing Spring 2016 Adam Shook.
CS520 Web Programming Introduction to Web Services Chengyu Sun California State University, Los Angeles.
Ext JS - Direct Bridging The Gap A DMSBT Presentation By Timothy Chandler.
Redmond Protocols Plugfest 2016 Jinghui Zhang Office Interoperability Test Tools (Test Suites and Open Source Projects) Software Engineer Microsoft Corporation.
Added Value to XForms by Web Services Supporting XML Protocols Elina Vartiainen Timo-Pekka Viljamaa T Research Seminar on Digital Media Autumn.
Windows Communication Foundation and Web Services
Windows Communication Foundation (WCF)
Distribution of functionality Webservice using WCF
Serialization.
Windows Communication Foundation and Web Services
CSCI-1680 RPC and Data Representation
Windows Communication Foundation (WCF)
Quiz Points 3 Rules Raise your hand if you know the question
Talking Between Services with gRPC
Presentation transcript:

Portable binary serialization, the Google way Protocol Buffers Portable binary serialization, the Google way

What problem? For high volume work, no ideal serializer: Xml/data-contracts: (relatively) expensive to process; large due to repeated tags Soap: as xml, but more BinaryFormatter (/NDCS): proprietary closed [non-]standard Bespoke: lots of work; lots of potential for error

What is protobuf? Protocol Buffers defines two things: A compact binary serialization format (pb) A text-based descriptor language (.proto) Implementation specific: Runtime serialization library / code .proto parser / code generator protobuf-net is one of 3 in-progress efforts for .NET

What is a .proto? Yet another descriptor language... message Test1 { required int32 a = 1; } message Test3 { optional Test1 c = 3; service SearchService { rpc Search (SearchRequest) returns (SearchResponse);

So why bother? Compact: Portable: Fast: no string field names (contrast: XmlSerializer; DataContractSerializer) uses variable-length encoding where sensible minimal framing overhead Portable: cross-platform (contrast: BinaryFormatter) extensible (roundtrip-safe) (protobuf-net works with .NET 2.0; .NET 3.0; CF 2.0; CF 3.0; Silverlight 2.0; Mono) Fast: efficient to parse Optimised for common scenarios, but doesn’t support every complex worst-case scenario.

Look familiar? Closely related to data/service-contracts [DataContract] public class Test1 { [DataMember(Name="a", Order=1, IsRequired=true)] public int A {get;set;} } public class Test3 [DataMember(Name="c", Order=3, IsRequired=false)] public Test1 C {get;set;} [ServiceContract] public interface ISearchService [OperationContract] SearchResponse Search(SearchRequest request);

Demo

Where can we use it? Communications Persistance Remoting WCF Sockets / http (proto-rpc?) Persistance File system Database BLOB SQL/CLR?

Existing data? Works with common frameworks: WCF LINQ-based data-contracts XmlType / XmlElement Can be used to replace, or in parallel with, standard serializers: ISerializable IXmlSerializable Custom attributes for full control Can also be specified at the class (rather than member) for use with partial classes