1. SeCoGIS 20081 Managing Sensor Data of Urban Traffic M. Joliveau 1, F.De Vuyst 1, G. Jomier 2, C.M. Bauzer Medeiros 3 ACI Masses de Données CADDY (2003-2007) (1) MAS, Ecole Centrale de Paris (2) LAMSADE, Université Paris-Dauphine (3) IC, UNICAMP

2. SeCoGIS 20082 Goals  Urban road traffic analysis congestions  Query the past behavior  Foresee the future behavior  Show understandable résults (Google Maps)

3. SeCoGIS 20083 Outline  Received Data  Exploratory studies  Deeper Analysis  Work to do  Concluding remarks (Google Maps)

4. SeCoGIS 20084 Data about the system to be studied - Graph with hundreds of sensors - Flow rate, occupancy rate, 3’ - States: fluid (0) / congestion (1) - Annotations From INRETS

5. SeCoGIS 20085 Mass of Data Sensor number (I) Day number (J) Number of measures in a day (K) High rate of missing data Bad quality of data Size order of the volume O(10 9 ) as I, J, K : O(10 3 )

6. SeCoGIS 20086 Exploratory study  Temporal view  Space-time view : dynamic vizualization of the sensor state map Flow rateOccupancy Rate Hours 0->24h

7. SeCoGIS 20087 Traffic States: fluid/congestion It appears : 2 states are not enough to characterize the dynamic behavior of the system Urban Traffic Spatio-temporal patterns

8. SeCoGIS 20088 Space-Time Vizualization flow rate x Time y

9. SeCoGIS 20089 Analysis of temporal series Extract of one week for a sensor among 400 Regularity of the human activity generating traffic

10. SeCoGIS 200810 Schema of Data Base for Analysis sensor-id Sensors day-id Days hour-id Hours annotation-id Annotation weather-id Weather sensor-id day-id hour-id annotation-id weather-id Traffic flow rate occupancy rate traffic state

11. SeCoGIS 200811 Symbolic representation of sets of temporal series  Symbol = label associated to a class  reduction of size and intelligibility  Class  identification of typical behavior, detection of atypical behaviors Episod partition Symbol Alphabet Symbolic Representation

12. SeCoGIS 200812 Plan  Received Data  Exploratory studies  Deeper Analysis STPCA Continuous Traffic State Variable  Concluding remarks

13. SeCoGIS 200813 STPCA Spatio-Temporal Principal Component Analysis Goal : data representation in a reduced number of  spatial dimensions => sensors  temporal dimensions => daily instants Result : Data projection simultaneously on the first spatial and temporal eigenmodes 1st experiment : Flow rate (Monday to Friday) for a family of reliable sensors

14. SeCoGIS 200814 Spatial Reduction X d (complete) matrix of daily realizations element x i,t, i sensor, t instant, d day T number of instants by day N number of days I number of sensors Y assembles horizontally N matrices X d : Y = col (X 1, X 2,......,X N )

15. SeCoGIS 200815 Sensors Number Number of Measure Instants Daily Data Matrix Y for spatial reduction

16. SeCoGIS 200816 Spatial Reduction Y assembles horizontally N matrices X d : Y = col (X 1, X 2,...,X N ) Each line is a temporal serie for 1 sensor Singular value decomposition of Y Spatial correlation matrix: M S = YY T Eigenvalues l 1 >= l 2 >=... l KM Eigenvectors (F k ) for k = 1…K M

17. SeCoGIS 200817 Spatial Reduction Spatial correlation matrix M s = YY T Eigenvalues: λ 1 ≥ λ 2 ≥... λ KM Eigenvectors: Ψ k for k = 1…K M P matrix of the K first eigenvectors Ψ k P = col (Ψ 1, Ψ 2,... Ψ K ) for K<< K M

18. SeCoGIS 200818 Spatial Reduction Estimate X’ d of each realization X d : X’ d = P P T X d K reduced spatial order Reduced order matrix : X r = P T X contains latent (hidden) variables of X size : K * T (T instants) If T is large, the dimension of the reduced order representation is too large

19. SeCoGIS 200819 Temporal Reduction Z assembles vertically N day realizations X d : Z = row (X 1, X 2,...,X N ) one colon corresponds to one instant t one line corresponds to one sensor i for one day d the data of one day d are grouped I* N lines

20. SeCoGIS 200820 Sensors Number Number of Measure Instants Daily Data Matrix Z for temporal reduction

21. SeCoGIS 200821 Temporal Reduction Z assembles vertically N day realizations X d : Z = row (X 1, X 2,...,X N ) Singular value decomposition of Z Temporal correlation matrix M t = Z T Z Eigenvalues μ 1 ≥ μ 2 ≥... μ LM Eigenvectors (Φ l ) for l = 1, 2…L M Q matrix of the L first eigenvectors Φ l Q = col (Φ 1, Φ 2,...Φ L ) for L << L M

22. SeCoGIS 200822 Temporal Reduction Estimate X’ for each realization X: X’ = X Q Q T Reduced order matrix: X r = XQ contains the latent variables of X size : I *L If I (space : number of sensors) is large the dimension of the reduced order representation is too high

23. SeCoGIS 200823 Results of temporal component analysis The 6 first temporal modes (ACP-t) -order :colon- define the matrix Q (J r =6)

24. SeCoGIS 200824 Reduction: projection on the first temporal mode Flow rates, 1 work day, 6 sensors - Observed flow rate - Projection on the 1rst temporal mode temps Capteur 1 Capteur 3 Capteur 5 Capteur 2 Capteur 4 Capteur 6 Time Sensor 1 Sensor 2 Sensor 3 Sensor 4 Sensor 5 Sensor 6

25. SeCoGIS 200825 Spatio-Temporal Reduction Combines spatial and temporal analysis new estimate of each realization X X’ = PP T XQQ T Reduced order matrix: X r =P T XQ contains the latent variables of X size K*L

26. SeCoGIS 200826 Cumulative Energy Spatial correlation matrix Eigenvalue Index Cumulative Energy Temporal correlation matrix

27. SeCoGIS 200827 Sensor 1Sensor 2 Sensor 3 Sensor 4 Sensor 5 Sensor 6Sensor 7 Sensor 8 Sensor 9 Sensor 10Sensor 11 Sensor 12 Sensor 13 Sensor 14Sensor 15 Sensor 16 Work days K=3, L=3

28. SeCoGIS 200828 Mean Direct Error Standard Deviation Reduced-order Matrix Size Mean Direct Error Standard Deviation Reduced-order Matrix Size Mean Direct Error Standard Deviation Reduced-order Matrix Size

29. SeCoGIS 200829 g Sensor 1Sensor 2Sensor 3Sensor 4 Sensor 5 Sensor 9 Sensor 13 Sensor 6 Sensor 10 Sensor 14 Sensor 7 Sensor 11 Sensor 15 Sensor 8 Sensor 12 Sensor 16 Chrismas Day K=3, L=3

30. SeCoGIS 200830 Error Distribution Function Sensors Number of Sensors

31. SeCoGIS 200831 Error Distribution Function Days Number of days

32. SeCoGIS 200832 Plan  Received Data  Exploratory studies  Deeper Analysis STPCA Continuous Traffic State Variable  Concluding remarks

33. SeCoGIS 200833 Generation of 7 new traffic states using analysis in phase space Saturé Fluide Grande circulation Occupancy Rate Flow Rate

34. SeCoGIS 200834 Continuous traffic state variable Occupancy rate Throughput

35. SeCoGIS 200835 Sensor 1 Time (hour) Flow Rate (nb vehicles) Occupancy Rate (%) Circulation State (%)

36. SeCoGIS 200836 State NameSymb. StateNum. SymbolE value at tDeriv. Sign in t Calm Negative Very high level circ. Saturation level 1 High level circul. Saturation level 2 Saturation level 3 Positive Back to Calm New circulation states

37. SeCoGIS 200837 Dynamic Visualization of the Traffic State Fluid Congestion Animation : spatio-temporal patterns appear

38. SeCoGIS 200838 Other results  Missing Data  STPCA for state variables  Spatio-temporal patterns  See Marc Joliveau ‘s PhD Thesis

39. SeCoGIS 200839 Work to be done  Enrich the datawarehouse with summaries, GIS, results of STPCA…  Symbolic spatio-temporal analysis  Adaptation to evolution  Visualization, user interaction  Refinement on types of days, episodes  Datawarehouse : queries

40. SeCoGIS 200840 Concluding Remarks  Reduction : from data masses to intelligible and manipulable elements  Generic Approach For spatio-temporal analysis of flow systems, described by data coming from a network of static georeferenced sensors with diffuse sources and wells

41. SeCoGIS 200841 Future Prospects  Data coming from embarked sensors  Go farther in spatio-temporal reduction