Image Processing Image Processing Windows HPC Server 2008 HPC Job Scheduler Dryad DryadLINQ Machine Learning Graph Analysis Graph Analysis Data Mining.NET Applications … Windows HPC Server 2008

Processing vertices Channels (file, fifo, pipe) Inputs Outputs

6

Query plan LINQ query Dryad logs where select Automatic query plan generation Distributed query execution by Dryad var logentries = from line in logs where !line.StartsWith("#") select new LogEntry(line);

\\HPCMETAHN01\XC\output\520a0fcf\Part

Go through logs and keep only lines that are not comments. Parse each line into a new LogEntry object. Go through logentries and keep only entries that are accesses by jvert. Group jvert accesses according to what page they correspond to. For each page, count the occurrences. Sort the pages jvert has accessed according to access frequency.

logentries user logs accesses htmAccesses output

List of files to be shipped to the cluster Vertex definitions

public static Vector NearestCenter(Vector v, IEnumerable centers) { return centers.Aggregate((r, c) => (r - v).Norm2() < (c - v).Norm2() ? r : c); } public static IQueryable Step(IQueryable vectors, IQueryable centers) { return vectors.GroupBy(point => NearestCenter(point, centers)).Select(group => group.Aggregate((x,y) => x + y) / group.Count()); } var vectors = PartitionedTable.Get ("vectors.pt"); IQueryable centers = vectors.Take(100); for (int i = 0; i < 10; i++) { centers = Step(vectors, centers); } centers.ToPartitionedTable (“centers.pt”); public class Vector { public double[] entries; [Associative] public static Vector operator +(Vector v1, Vector v2) { … } public static Vector operator -(Vector v1, Vector v2) { … } public double Norm2() {…} }

Built by Developers for Developers….

© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.