De-mystifying Google’s hottest binary protocol Prasanna Kanagasabai Jovin Lobo

About us :  Prasanna Kanagasabai :  Security ThoughtWorks  Member of null- The Open Security Community.  Author of IronSAP a module over IronWASP.  nullcon-Delhi, Clubhack, IIT Guwahati and various null meetups.  Jovin Lobo :  Associate Aujas Networks  Member of null- The Open Security Community.  Author of GameOver – Linux distro for learning web security.  Spoken at nullCon, GNUnify before.

Agenda  Introduction.  Anatomy of Protobufs  Defining Message formats in.Proto files.  Protobuf compiler  Python API to read write messages.  Encoding Scheme  Problem Statement.  Decoding like-a-pro with IronWasp ‘Protobuf Decoder’.

Introduction:  Protocol Buffers a.k.a Protobufs :  Protobufs are Google's own way of serializing structured data.  Extensible, language-neutral and platform- neutral.  Smaller, faster and simpler to implement.  Java, C++ and Python

Anatomy:  Over view :

Defining a.Proto file.  #> less Example.proto message Conference { required string conf_name = 1 ; required int32 no_of_days = 2 ; optional string = 3 ; } // * 1,2,3 are unique tags. These are used by the fields in binary encoding. * For optimization use tags from 1-15 as higher nos. will use one more byte to encode.

Compiling  Syntax:  protoc –I=$_input_Dir --python_out=$_out_Dir $_Path_ProtoFile  Eg:  protoc –I=. --python_out=. Example.proto This will generate a Example_pb2.py file in the specified destination directory.

$ProtoFile_pb2.py  The Protobuf compiler generates special descriptors for all your messages, enums, and fields.  It also generates empty classes, one for each message type:  Eg:

Reading and writing messages using the Protobuf binary format :  SerializeToString()  serializes the message and returns it as a string.  ParseFromString(data)  parses a message from the given string.

Demo: Protobuf… how it wrks

Encoding.  example2.proto message Ex1 { required int32 num = 1; // field tag }  Code snippet: obj = example2_pb2.Ex1(); obj.num = 290; // field value obj.SerializeToString();  Output : 08 A2 02 #hex #binary

Problem statement.

This is what freaked him out 08 A

Lets Decode it..  Step 1 :  Find the wire type.  Step 2:  Find the field number.  Step 3:  Find the field tag.

Step1: finding wire type.   To find wire type take the first byte:   [0] Drop MSB from First byte.  The last 3 bits give wire type.  Wire type is 000  type = 0 is Varint.

Wire types

Step 2: Field tag.  What we already have is  Now we right shift value by 3 bits and the remaining bits will give us the field tag.   ‘0001 ‘ i.e. ‘ 1’  So we get the field tag = 1

Step 3: Find the field value   We drop the 1 st byte   Drop the MSB’s from each of these bytes    Reverse these bytes to obtain the field value.   i.e = 290  So we finally get the value of the field = 290.

So we successfully decoded  example2.proto message Ex1 { required int32 num = 1; }  Code snippet: obj = example2_pb2.Ex1(); obj.num = 290; obj.SerializeToString();  Output : 08 A2 02 #hex #binary  We successfully Decoded Value : “290”

Demo : Lets do this live

Automating all this with IronWasp Protobuf Decoder:  About IronWasp :  IronWasp is an open-source web security scanner.  It is designed to be customizable to the extent where users can create their own custom security scanners using it.  Author – Lavakumar Kuppan  Website :

ProtoBuf Decoder

Road Map for Protobuf Decoder

Hmmm … Decoding ……

Any Questions ? Done … It says ……

Any Questions ? Done … It says ……

Thank You