Counting windows Project participants (in alphabetical order): Akif Durdu Middle East Technical University, Turkiye Viktor Jónás Budapest Polytechnic, Hungary Csaba Pintér University of Szeged, Hungary Péter Rieger Budapest Polytechnic, Hungary Umut Tilki Middle East Technical University, Turkiye

Project Overview Main task was to count windows on the photo of a building Input: - Image that the user wants to count windows in - A length of the diagonal of an average sized window (user selects it on the input picture) Output: - Number of found windows - Output picture containing the original picture and the contours of the found windows

Overview: Preprocessing Preprocessing Detecting lines with Hough transformation Detecting lines with Hough transformation It will define boxes (creates separate quads) It will define boxes (creates separate quads) unifying “similar” boxes Classify remaining boxes (windows non- windows) Classify remaining boxes (windows non- windows)

Preprocessing: Converting image to grayscale Converting image to grayscale Normalizing Normalizing Median filter (blurring) Median filter (blurring) Sobel operator (finding edges) Sobel operator (finding edges) Opening Opening

Preprocessing 1 – Converting to grayscale: Color information is not so important during preprocessing Color information is not so important during preprocessing Faster Faster Easier to handle Easier to handle

Preprocessing 2 – Normalizing Greater contrast Greater contrast More determining edges More determining edges

Preprocessing 3 – Median filter Blurs the unimportant edges (e.g. gutters, window frames (inner+outer=2)) Blurs the unimportant edges (e.g. gutters, window frames (inner+outer=2)) Still preserves the other edges Still preserves the other edges Disadvantage: rounds corners (Hough can compensate) Disadvantage: rounds corners (Hough can compensate) Parameters: Parameters: repetitions:3 repetitions:3 window size:square root of the user input window diagonal window size:square root of the user input window diagonal

Preprocessing 3 – Median filter

Preprocessing 4 – Sobel operator Finds the edges Finds the edges Prepares the picture for Hough transform Prepares the picture for Hough transform

Preprocessing 5 – Opening Dilation + erosion Dilation + erosion Reduces remaining noise (unifies broken edges) Reduces remaining noise (unifies broken edges) Makes possible double frame edges disappear Makes possible double frame edges disappear

Preprocessing 5 – Opening

Detecting lines with Hough transformation: Applying Hough transformation Applying Hough transformation Finding local maxima (finding the important lines) Finding local maxima (finding the important lines) Applying inverse Hough transformation (projecting found lines back) Applying inverse Hough transformation (projecting found lines back)

Hough 1 – Applying Image space -> Hough space transformation Image space -> Hough space transformation Strong lines with higher intensity Strong lines with higher intensity

Hough 2 – Finding local maxima Separating dominant lines Separating dominant lines Window size: Window size: 3*sqrt(UI diagonal)

Hough 3 – Inverse Hough transformation Hough space -> image space [inverse Hough formulas] Hough space -> image space [inverse Hough formulas] Projecting dominant lines back Projecting dominant lines back It defines boxes (general quads) It defines boxes (general quads) ideal actual

Unifying “similar” boxes Classify image to boxes (floodfill) Classify image to boxes (floodfill) computing statistics computing statistics unifying “similar” boxes (similar their statistics have little difference) unifying “similar” boxes (similar their statistics have little difference)

Unifying 1 – Flood fill It classifies image to boxes It classifies image to boxes Each class is a general quad Each class is a general quad it is done with a simple floodfill until there are any black spots remaining it is done with a simple floodfill until there are any black spots remaining the color values of the areas represent the classes the color values of the areas represent the classes

Unifying 2 – Statistics They are used to represent a kind of similarity measure They are used to represent a kind of similarity measure Statistics: mean and variance of each color channels some more statistics: center coordinates, area (for an easier determination of neighbourhood) Statistics: mean and variance of each color channels some more statistics: center coordinates, area (for an easier determination of neighbourhood) The sample variance Eqn. The sample mean Eqn.

Unifying 3 – The unifying itself we must decrease the number of classes radically we must decrease the number of classes radically the reamaining classes may be the reamaining classes may be whole windows (we hope ) whole windows (we hope ) larger homogeneous areas (roof, grass, sky …) larger homogeneous areas (roof, grass, sky …)

Unifying 3 – The unifying itself The unifying algorithm The unifying algorithm put each class into a queue while (there is a class in the queue) { while (there is a similar neighbor) { unify them compute new statistics } remove from queue }

Classifying remaining boxes we must decide if an area represents a window or not we must decide if an area represents a window or not we do this according to the user input “ideal” window we do this according to the user input “ideal” window the idea: if the ratio of edges or area is reasonable, let’s consider it to be a window, else it possibly belong to the class of non-windows the idea: if the ratio of edges or area is reasonable, let’s consider it to be a window, else it possibly belong to the class of non-windows

The result

THE END Thank YOU Have a nice day