Presentation is loading. Please wait.

Presentation is loading. Please wait.

Workshop on HPC in India Prof: N.Pradhan Department of Psychopharmacology National Institute of Mental Health & Neurosciences, Bangalore 560029 (India)

Similar presentations


Presentation on theme: "Workshop on HPC in India Prof: N.Pradhan Department of Psychopharmacology National Institute of Mental Health & Neurosciences, Bangalore 560029 (India)"— Presentation transcript:

1 Workshop on HPC in India Prof: N.Pradhan Department of Psychopharmacology National Institute of Mental Health & Neurosciences, Bangalore 560029 (India) nprnimhans@gmail.com ATIP 1 st Workshop on HPC in India @ SC-09 HIGH PERFORMANCE COMPUTING AT NIMHANS AND ITS IMPACT ON MENTAL HEALTH RESEARCH IN INDIA.

2 Introducing to the Need for HPC in Mental Health Research Brief Summary of Research Areas using HPC at NIMHANS How we built our HPC and its performance Conclusion HIGH PERFORMANCE COMPUTING AT NIMHANS AND ITS IMPACT ON MENTAL HEALTH RESEARCH IN INDIA. N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-092

3 Introducing to the Need for HPC in Mental Health Research MENTAL HEALTH IS MULTI-DIMENSIONAL ISSUES RANGE FROM MOLECULE TO PSYCHOLOGY, SOCIAL SCIENCES, LAW AND HUMAN RIGHTS. MENTAL HEALTH INSTITUTES / NEUROSCIENCE INSTITUTES IN INDIA NONE OTHER THAN NIMHANS DEAL WITH HUMAN BRAIN & BEHAVIOUR UNDERSTANDING HUMAN BEHAVIOUR HPC “MOLECULAR AND SYSTEMS LEVEL” IN HEALTH AND DISEASE STATES: Diagnosis, Monitoring, Therapeutic Response Evaluation and Prognostic. N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-093

4 NIMHANS AND HPC HUMAN BRAIN OCCUPIES THE CENTER STAGE EVERYWHERE IT IS TOO COMPLEX Immense Synaptic Connectivities Understanding Brain Functions Remains Descriptive Medical Observation with Brain Lesions. (No Definite Model for Any Function) THE HPC NEED HIGH RESOLUTION MAPPING Both Structural and Functional: Example Epileptic Surgery MOLECULAR APPROACH TO DIAGNOSIS AND THERAPY Example: Myopathies, Degenerative disorders N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-094

5 NIMHANS AND HPC IT IS IMPLIED THAT WHILE WE ADDRESS THE ISSUES OF THE BRAIN HPC IS AN IMPORTANT TOOL OF INVESTIGATIVE NATURE BRAIN DOES NOT COMPUTE : WE MAKE A CARICATUTRAL MODEL OF THE BRAIN BEHAVIOURAL SCIENCES AND MENTAL HEATH HAVE BEEN KEPT AWAY FROM SUPERCOMPUTING. SUPERCOMPUTERS ARE EXPENSIVE AND NONETHELESS INFINITESIMALLY SMALL IN COMPUTING POWER TO ADDRESS ANY QUESTION RELATED TO BRAIN AND ITS BEHAVIOUR. IT NOT POSSIBLE TO EMULATE HUMAN PERCEPTUAL- COGNITIVE BEHAVIOUR AND EMOTIONS IN ANY SUPERCOMPUTER HOWEVER REFINEMENT OF CERTAIN ROUTINE CLINICAL PROCEDURES NEED HPC NOW (DATA CENTERS AND HMS) N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-095

6 Summary of Research Areas using HPC at NIMHANS PURPOSE OF BUILDING OUR HPC MASSIVE SCALE BRAIN SIGNAL PROCESSING (BRAIN DYNAMICS) fMRI MAGE PROCESSING TRAINING AND RESEARCH IN MOLECULAR PHARMACOLOGY BIOINFORMATICS AND MOLECULAR DYNAMICS NEURAL NETWORKS SIMULATIONS N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-096

7 BRAIN SIGNAL PROCESSING MULTICHANNEL BIOMEDICAL SIGNALS 32 – 256 CHANNELS OF DATA SAMPLING 256 TO 40K DURATION 10 SEC TO 8 HOURS TO DAYS Total Data size could in GBs MEG may be added this Year EPILEPSY MONITORING ICU (COMA, STROKE ETC) ANAETHESIA SLEEP ELECTRO-CONVULSIVE THERAPY AUTONOMIC DYSFUNCTIONS N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-097

8 BRAIN SIGNAL PROCESSINGS THERE IS NO LIMIT TO HOW MUCH OF MATHEMATICS BE APPLIED TO UNDERSTAND THE PATTERN FEATURES OF EEG IN VARIOUS NEUROLOGICAL AND PSYCHIATRIC CONDITIONS The Fundamental Requirements SOURCE localization Functional Characterization (Multimodal) Modeling Simulation In Conjunction with SPECT MRI / fMRI N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-098

9 NIMHANS AND HPC (APPLICATIONS) N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-099

10 BRAIN SIGNAL PROCESSING BRAIN DYNAMICS PRE-PROCESSING: FILTRATION AND ARTIFACT REMOVAL IN BAND NOISE CANCELLATION POST -PROCESSING: LINEAR METHODS NONLINEAR METHODS N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0910

11 NIMHANS AND HPC APPLICATIONS NONLINEAR DYAMICAL INVARIANTS N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0911

12 NIMHANS AND HPC APPLICATIONS NONLINEAR DYAMICAL INVARIANTS N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0912

13 NIMHANS AND HPC APPLICATIONS NONLINEAR DYAMICAL INVARIANTS N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0913

14 NIMHANS AND HPC APPLICATIONS NONLINEAR DYAMICAL INVARIANTS N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0914

15 NIMHANS AND HPC APPLICATIONS NONLINEAR DYAMICAL INVARIANTS N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0915

16 NIMHANS AND HPC APPLICATIONS NONLINEAR DYAMICAL INVARIANTS N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0916

17 NIMHANS AND HPC (APPLICATIONS) N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0917

18 NIMHANS AND HPC (APPLICATIONS) fMRI Image Processing N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0918

19 NIMHANS AND HPC (APPLICATIONS) fMRI Image Processing N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0919

20 NIMHANS AND HPC (APPLICATIONS) fMRI Image Processing N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0920

21 NIMHANS AND HPC (APPLICATIONS) IMAGING BRAIN ACTIVITY A 3-D View of the Brain Imaging software combines data from several imaging technologies to create an interactive 3-D map of the brain. Reading Thoughts with Brain Imaging Researchers use fMRI to determine the contents of short-term memory. fMRI and Yoga N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0921

22 NIMHANS AND HPC (APPLICATIONS) MOLECULAR DYAMICS NMDA N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0922

23 NIMHANS AND HPC (APPLICATIONS) MOLECULAR DYAMICS α-synuclein N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0923

24 NIMHANS AND HPC (APPLICATIONS) MOLECULAR DYAMICS GAD N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0924

25 NIMHANS AND HPC (APPLICATIONS) MOLECULAR DYAMICS GAD N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0925

26 NIMHANS AND HPC (APPLICATIONS) MOLECULAR DYAMICS INHA-NADH N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0926

27 NIMHANS AND HPC (BUILDING THE HPC) IBM server 326 16 Nodes N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0927

28 NIMHANS AND HPC (BUILDING THE HPC) SUN V40Z TWO Nodes N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0928

29 NIMHANS AND HPC (BUILDING THE HPC) SUN V40Z TWO Nodes N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0929

30 THE HPC  The Sever In place is Cluster Built on Octal SMP Servers of HP DL 785 (8 No)  EACH NODE contained (In addition to Standard features)  8x 8384 Opteron 2.7 GHZ Quad Core Processors  128 GB RAM (Opeating at 533 GHZ –Fully Populated DIMMS)  4x 300GB highest supported speed SAS HDD  1xDVD ROM,  IB Inter-Conncect: Infiniband Card 4xDDR PCIe-dual port Con-X IB Card.  Manually Clustered without any Clustering Kit or software on Fedora 10 (Kernel 2.6.27)  Non-Reducing Loop Topology. N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0930

31 NIMHANS AND HPC (BUILDING THE HPC) HP DL875 8 NODES N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0931

32 NIMHANS AND HPC (BUILDING THE HPC) HP DL875 8 NODES N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0932

33 NIMHANS AND HPC (BUILDING THE HPC) ALPHA –PROCESSOR-NON Availability which made us look for other Processors CHOICE: (COST FACTOR ONLY) BUILDING USING THE AMD OPTERON SIMPLEST WAY - TOTAL OPEN SOURCE STARTING FEDORA 6 through FEDORA 10 TECHNIQUE: DETECT HARDWARES MAKE IB OVER IP ALL NODES ARE ON ONE SUBNET OPENSM - Infiniband subnet manager and administration (SM/SA) Opensm 3.3.2 TO SCAN IB HARDWAREFABRIC AND MAKE SUBNET UP LAMBOOT PROCESS to make Resources available as the user desires. LAM MPI 7.1.5b (LAM-MP connects well on Open IB) N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0933

34 NIMHANS AND HPC (APPLICATIONS)  Development of HPC step Wise  Vector Length of the Test problem to be decided  Processor Core status: Core to Core SMP Steams are synchronous  Physics of Problem. The length of the iterations in, femto, Pico seconds time steps to Nano Seconds  Path Modulation by streaming data in Multi Channel Mode through Dual Path IB Interconnect  Core path structure of Folding and Unfolding in Protein complexes  Protein Dock to Undock Rendezvous iterations in multiple data inputs and Outputs  Number of the Compute Cores in Cycle Change -256.  Data analysis of foot prints in Molecular Motion. N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0934

35 THE BENCHMARK The MD Simulation benchmark scaling is nearly linear and 100%, despite other users running jobs concurrently. ONE NODE: 32 Computing core:  NODE (s) Real (s) (%)  Time: 3398.600 3399.000 100.0  56:38  (Mnbf/s) (GFlops) (ns/day) (hour/ns)  Performance: 34.273 1.207 0.254 94.387  FOUR NODES: 128 Computing Cores (IB Connect)  Average load imbalance: 4.2 %  Part of the total run time spent waiting due to load imbalance: 2.2 %  Parallel run - timing based on wallclock.  NODE (s) Real (s) (%)  Time: 696.000 696.000 100.0  11:36  (Mnbf/s) (GFlops) (ns/day) (hour/ns)  Performance: 167.316 5.880 1.242 19.329   EIGHT NODES: 256 Computing Cores (IB Connect)   Average load imbalance: 7.9 %  Part of the total run time spent waiting due to load imbalance: 3.8 %  Parallel run - timing based on wallclock.   NODE (s) Real (s) (%)  Time: 370.000 370.000 100.0  6:10  (Mnbf/s) (GFlops) (ns/day) (hour/ns)  Performance: 314.768 11.060 2.336 10.276 N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0935

36 NIMHANS AND HPC (APPLICATIONS) CONCLUSION WE have shown HPCNEED HOW TO BUILD What is its Performance LOW COST HPC can be built manually using Total Open Source. One need not pay extraordinary price for proprietary interconnects. N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0936

37 THE HPC  The Sever In place is Cluster Built on Octal SMP Servers of HP DL 785 (8 No)  EACH NODE contained (In addition to Standard features)  8x 8384 Opteron 2.7 GHZ Quad Core Processors  128 GB RAM (Opeating at 533 GHZ –Fully Populated DIMMS)  4x 300GB highest supported speed SAS HDD  1xDVD ROM,  IB Inter-Conncect: Infiniband Card 4xDDR PCIe-dual port Con-X IB Card.  Manually Clustered without any Clustering Kit or software on Fedora 10 (Kernel 2.6.27)  Non-Reducing Loop Topology. N PRADHAN ATIP 1 st Workshop on HPC in India @ SC-0937


Download ppt "Workshop on HPC in India Prof: N.Pradhan Department of Psychopharmacology National Institute of Mental Health & Neurosciences, Bangalore 560029 (India)"

Similar presentations


Ads by Google