Say Hello to my Little Friend - Fedora Messaging Infrastructure

What is Messaging? A standard infrastructure for routing useful information from one place to the next E-mail is a specific messaging system optimized for sending human readable text We need a way to send computer parsable messages Linux uses D-Bus as its desktop messaging protocol

What is AMQP? Advanced Message Queuing Protocol An industry standard for building messaging infrastructure used mainly by banks Provides routing, ACL mechanisms, messaging patterns and realtime capabilities Provides standard data type marshaling

What is QMF? Qpid Management Framework A protocol built on top of AMQP Provides a complete object and event system Events Objects Properties Statistics

What is Qpid? Apache's implementation of the AMQP and QMF standards Both Java and C++ brokers We use the C++ broker as it is the only one to support QMF at the moment Python, Ruby and C++ bindings are available yum search qpid

Why not D-Bus (insert favorite messaging protocol here)? Distributed messaging – we want upstream to get our data and visa versa While D-Bus was built for simplicity and the desktop it doesn't have the flexibility needed for distributed internet messaging ReST interfaces are here to stay but they are pull only and not very efficient AMQP has push capabilities, packetted data and interesting optimization paths

Why does Fedora Need Messaging? E-mail spam/notification isn't very useful for most people A computer parsable messaging format means we can display notifications in a way that is most useful for each user It also means we can automate processes federate in the future to get notifications from upstream sources

How do we plan on using Messaging?

Security Domain Architecture Publishers send events to the main Fedora Infrastructure broker Subscribers listen for events within their security domain's broker Forwarding rules specify what messages can be forwarded into what security domain Firewall rules, browser domain security and ACLs mean clients can only talk within their security domain

Security Domain Architecture(cont.) Forwarding links can be severed without disrupting applications. A FAS ACL enabled security domain would allow monitoring abuse per user and shutting out their account if suspicious activity is detected Domains that are using up too much bandwidth can be limited while other domains can be given priority for time sensitive channels

Security Domain Issues Slightly more complex rules need to be maintained New features such as QMF don't fully support federated brokers (but this is on the development timeline)

AMQP Model Server +-------------------------------+ | Virtual host | | +-----------------------+ | | | | | +-------------+ | | +-----------+ | | | Publisher | ----------> | Exchange | | | | Application | | | +-----+-----+ | | +-------------+ | | | | | | | | | | | | | Binding | | | | \|/ | | +-------------+ | | +---------+ | | | Consumer | <----------- | Message | | | | Application | | | | Queue | | |

Message Flow +-------------+ +-------+ +-------------+ +-------+ | Publisher | -----------------> |Message| | application | +---+---+ +-------------+ | | +---------+ |Exchange | +----+----+ +------------+------------+ | | | Message Message Message Queue Queue Queue +-------------+ +-------+ +-------+ +-------+ | Consumer | +-------+ +-------+ +-------+ | application | <---- |Message| +-------+ +-------+

Payload Formats AMQP only defines the routing and queue manipulation parts of a complete protocol Content is just a binary blob requiring some sort of explicit contract between the client and the server as to how it is formatted Options are spin our own or QMF

Spin our own w/ Publish/Subscribe pattern Use the amq.topic exchange which defines a routing key binding org.fedoraproject.# would be our routing key (e.g. koji would use org.fedoraproject.koji as its routing key) Simplicity is advantageous here Works with federation Doesn't define an object model for expanded functionality in the future

Spin our own w/ JSON Payload Pros Almost every language has a JSON library We already use this for ReST calls Cons Extra overhead JSON is not processed securely in every library Need to receive the whole JSON string before parsing can be done

Spin our own w/AMQP type lib Pros Use the AMQP library to marshal and demarshal data Fairly compact binary format Cons Better than the JSON format but still not a full object model Extensive type system can be confusing No tools to debug and not human readable

QMF Pros Complete object model gives us methods, properties and statistics for future expandability Uses a subset of the AMQP type library for payload w/ tools to debug messages Compact payloads based on schemas so types aren't transmitted with every message We have the developers' ears and can help shape QMF to our needs

QMF Cons Extra handshake plus schema fetching means high startup costs gearing it to long running processes Schemas aren't federated between brokers yet Bindings for producers aren't quite ready or are very low level Complexity of object system is an overhead we don't need just yet

We need to define use cases Koji Bodhi FAS PackageDB Package CVS Bugzilla Upstream etc.

Koji Events Build Started Build Completed Build Failed Tasks? - e.g. do we want higher level events or do we want to stick closer to the actual bare bones architecture?

Bodhi Events Update Requested Update Pushed Update Canceled Karma Incremented Karma Decremented

FAS Events User Added User Removed Group Added Group Removed User Added to Group User Removed from Group

PackageDB Events ACLs Changed Package Added Package Orphaned Package Removed

Package CVS Events Commit Made Repo Created Branch Created

Bugzilla Events New Bug Bug Updated Bug Closed

Upstream Events Package Released New Project