1 Client API Goals: evolvable, easy to use Design decision: –event-driven, non-blocking programming model –Data items are immutable Main data structure: di_node

2 Control Flow cbk_fun(…); … di_fun(…, cbk_fun); … marshal parameters… … create callback data structure, cbk… di_register_callback(…); send(cbk, …); … recv(cbk, …); … use cbk to retrieve callback associated info… … invoke callback … … free data struct. cbk … Client recv(cbk, …); … execute command … send(cbk, …); // send reply // back DIM/PM/Directory/… This code is executed in the cl_select() loop

3 di_node Data item data structure: –Metadata associated/stored with every data item –Created when a data item is looked up Fields –id: globally unique identifier –name: data item name –type: bulk (e.g., file transfer), stream (live, VoD), … Each type will have different constraints All types present the abstraction of a byte-stream –size: data item size –owner / access rights –mode: access mode in which the data item has been opened (e.g., Read / Write / Delete) –constraints: min download speed, max download speed, etc Need to decide how to compute the mix/max rates –Others: coding-format: specifies how the content is coded tags: keywords associated to this data item (can be used for search) …

4 Core Client API Generic functions: –di_init(); –di_exit(); –di_select(); –di_lookup(); DI-centric functions (all take as an argument a data item) –di_free(); –di_open(); –di_close(); –di_read(); –di_seek(); –di_write(); Note: di_init(), di_exit(), di_select(), di_free() and di_close() are blocking functions; the rest are non- blocking

5 di_init() Synopsis: errcode = di_init(); Description: blocking function call that initialize the client context: –Find an i3 server, and use it to contact other system components, e.g., DIMS and PMs –Contact database if new profile, and eventually obtain the list of policies the client belongs to –For each policy contact the corresponding PM –Contact the DIMs corresponding to data items and chunks the client stores

6 di_exit(); Synopsis: errcode = di_exit(); Description: blocking function performing the following operations: –Inform relevant PMs and DIMs –Update database with latest statistics –Free data structures allocated by di_init();

7 di_select(); Synopsis: di_select(…..); Description: preserve semantics of the select() function call, but in addition handles our protocol messages

8 di_register_callback() Synopsis: di_register_callback(di_node, cbk_fun(), cl_data); Description: A callback is registered for every non- blocking di_*() function to signal the client when di_*() has been completed Fields: –di_node: data item for which the callback is registered, if any –cbk_fun(): callback function invoked upon completion of corresponding di_*(…); –cl_data: client specific data; passed back to client in cbk_fun(…) Note: this function doesn’t need to be part of API

9 di_lookup() (1/2) Synopsis: di_lookup(search_string, flags, timeout, lookup_fun(), cid); Description: function which returns a list of data items that meet the search criteria Fields: –search_string: string used to search for a data item; relevant only when an existing data item is accessed (i.e., mode = Read OR Delete) –flags: specify how the search string should be interpreted (e.g., file name, tags, …) –timeout: timeout associated with this function call; used to return an error code if the function doesn’t complete –lookup_fun: function to be invoked on callback –cid: unique identifier associated with this function call; used to distinguish among multiple di_open() issued in parallel

10 di_lookup() (2/2) Callback registration: di_register_callback(NULL, lookup_fun(), cid); –Note: in this case, cid represents cl_data Callback function: lookup_fun(cid, di_node_list, errcode); –di_node_list: list of id_nodes of the data items that match the “search_string” –errcode: error code associated with this function call Open issue: –Need to decide what to do when di_node_list is too large

11 di_free() Synopsis: di_free(di_node); Description: free di_node returned by di_lookup();

12 di_open() (1/2) Synopsis: di_open(di_node, mode, timeout, open_fun()); Description: open a di_node for read/write/delete. In the case of “read”, this function checks whether there is a DIM associated with the data item (i.e., there is a seed for the data item), and if yes allocates required buffers. Fields: –di_node: data item to be opened. –mode: Read / Write / Delete –timeout: timeout associated with this function call; used to return an error code if the function doesn’t complete –open_fun: function to be invoked on callback

13 di_open() (2/2) Callback registration: di_register_callback(di_node, open_fun(), NULL); Callback function: open_fun(di_node, errcode); –di_node: data item that has been opened –errcode: error code associated with this function call

14 di_close() Synopsis: di_close(di_node); Description: this function frees the buffers allocated by di_open(). The function is blocking (it is executed locally).

15 di_read() (1/2) Synopsis: di_read(di_node, start_byte, last_byte, timeout, read_fun()); Description: this function requests all bytes from the input stream between start_byte and last_byte. A callback is invoked when one of the following events happen: –The incoming buffer has received all required bytes –The “timeout” expires –The byte-stream ends before reaching last_byte –There is a gap in the byte sequence, e.g., due to packet loss in live streaming –The last byte in the buffer has been written, and the buffer is about to wrap-around

16 di_read() (2/2) Callback registration: di_register_callback(di_node, read_fun(), NULL); Callback function: read_fun(di_node, ptr, size, s_byte, errcode); returns a contiguous sequence of bits from the incoming stream between s_byte and s_byte+size –ptr: pointer in the buffer from where the client should read data –size: how many bytes should the client read –s_byte: the byte in the byte-stream stored at location “ptr”. If no bytes are lost, then s_byte coincides with the start_byte value passed in di_read(); Otherwise, s_byte > start_byte. Also s_byte+size <= last_byte –errcode: specifies whether the timeout has been triggered or if any of the data item constraints have been violated (e.g., receiving rate too low or too high) Note: in the case of file sharing and VoD no data should be lost, and thus s_byte=start_byte

17 di_write() (1/2) Synopsis: di_write(di_node, fd, timeout, write_fun()); Description: create a live stream or a data item to be distributed via VoD or file sharing Fields: –di_node: data item structure associated with the file. This is created using di_open(di_node, “w”, timeout, open_fun()); –fd: For live streaming, this is the descriptor of the video device. A DIM is created for this data item. In addition, the database and the data item directory are updated. For VoD and file sharing, this is the descriptor of the file containing the data item. The file is divided into chunks which are stored locally, and the information about the data item is stored in the database and the data item directory.

18 di_write() (2/2) Callback registration: di_register_callback(di_node, write_fun(), NULL); Callback function: write_fun(di_node, errcode);

19 di_delete() Synopsis: di_delete(di_node, timeout, delete_fun()); Description: delete data item; update database and data item directory. This function only applies to VoD and file-sharing data items Callback registration: di_register_callback(di_node, delete_fun(), NULL); Callback function: delete_fun(di_node, errcoe);