1. Practical work on NEMO Converting files
Training Workshop – Ostende – 1-4 March 2010
Practical work on NEMO Converting files
M. Fichaut

2. From ASCII data to XML files
Collection of
ASCII files
ODV
files
XML
CDI files
summary_CDI_NEMO.xml
Delivered with NEMO
SeaDataNet
CDI
CDI summary
CSV file
Coupling table
MIKADO
Export
Coupling table

3. NEMO installation
Open the install_nemo_1.4.4_windows.zip file (on the desktop)
Unzip it
Double click on launcher_nemo.bat
Follow the installation procedure
Don’t forget to add the NEMO shortcut on your desktop

4. Collection of ASCII files
Training data set
6 CTDs raw files under
SDN tools/Practical work/data/input files
ASCII files with measured parameters in column
Measured parameters = Depth, Temperature, Salinity, Fluorescence and Dissolved oxygen

5. First step : conversion to ODV with NEMO
Run NEMO to convert the files to ODV format.
Modify the settings of NEMO
Default directories for data set to
SDN tools/Practical work/data
SDN tools/Practical work/models
Enter your EDMO ID in the field below using the search function (right click)
Proceed the 4 steps of NEMO

6. NEMO – File description
Choose the type of file (cruise directory, cruise file, …)
Browse your PC to input the data files directory
Once the file is open in NEMO, have a look to its content
Choose file type : Profile
Choose the output format : ODV, one unique file for all stations or One file per station
Enter the Station header information
there is a specific character string at the end of each Station header : *END*
The Data termination indicator is EOF (End of file)
Validate the step when description is OK

7. NEMO – Station description
Station number is given on line “**Station”
Data type is ‘CTD stations’
Time is U.T.
Latitude format is NDD MM.mm
Longitude format is NDDD.MM.mm
Bottom Depth is not available
Input all the mandatory fields : station number, time, date, latitude, longitude
Use the Test function to verify your input
Validate the step when description is OK

8. NEMO – Data description (1)
Select the parameter list to use : P011 via P021
Add the parameters : P021 codes are
Depth : AHGT, Temperature : TEMP, Salinity : PSAL, Fluorescence : FVLT, Dissolved oxygen : DOXY, Input the units of the parameters (for salinity, use Dimensionless)
Add a text label to each parameter
Set Start and End positions for the parameters
Tell NEMO which code is the vertical reference

9. NEMO – Data description (2)
Format is mandatory
Describe the output format that you want
%6.1f : means 6 digits total (decimal point included), 1 digit for decimal, f for float (decimal number)
Use the contextual menu ‘Select the format’ for predefined format
Use the test fields to validate your input
Validate the step when description is OK
Save you model in the model directory

10. File conversion You will be asked to give :
A dataset reference (can be a cruise name, for example)
A name for the output file (all the stations in the same file) or name for the output directory (one station per file) in the SDN tools\Practical work\output directory
Have a look to the output files
Drag and drop the ODV files in ODV to test their conformity

11. Practical work on NEMO Generating CDI summary and coupling table
M. Fichaut

12. From ASCII data to XML files
Collection of
ASCII files
ODV
files
XML
CDI files
summary_CDI_NEMO.xml
Delivered with NEMO
SeaDataNet
CDI
CDI summary
CSV file
Coupling table
MIKADO
Export
Coupling table

13. Creation of the CDI-SUMMARY and the coupling table
Input the information in NEMO SETTINGS
Mapping
Click in the box Generate mapping for SeaDataNet download Manager
SeaDataNET CDI summary
Unfold the SeaDataNet CDI Summary block
Click in the box Generate SeaDataNet CDI summary
Fulfill all the mandatory fields (in red), using right click for the Search function

14. Run the file conversion
Same file than in the previous exercise
You will be asked to give :
A dataset reference (can be a cruise name, for example)
A name for the CDI summary file
A name for the output file (all the stations in the same file) or name for the output directory (one station per file)
An output directory prefix (for the coupling table). This prefix will be subtracted from the file name in the mapping table.
For example if
output file name = C:\username\NEMO\cruise_name\file_name
and output directory prefix = C:\username\NEMO
file name in the mapping table will be : cruise_name\file_name

15. CDI summary file
Convert it to an EXCEL file (for future use by MIKADO)
Open the CDI summary text file with EXCEL (field delimited by tabulations)
Rename the folder into : STATION
Save it under XLS or XLSX format, it will be used later on during the practical work on MIKADO

16. Coupling table Have a look at the coupling table
Menu : Coupling_table>Edit
It gives the link between the LOCAL_CDI_ID and the file in which the LOCAL_CDI_ID will be found by the download manager of SeaDataNet
MODUS = 1 if the LOCAL_CDI_ID is in a mono-station file
MODUS = 3 if the LOCAL_CDI_ID is in a multi-station file
One LOCAL_CDI_ID and one format is unique in the coupling table
Add and delete records
Export the coupling table in a flat file
Menu : Coupling_table>Export
Have a look at the flat coupling.txt file

17. Practical work on MIKADO Creation of XML files for CDI
Training Workshop – Ostende – 1-4 March 2010
Practical work on MIKADO Creation of XML files for CDI
M. Fichaut

18. Exercise 1
CDI Automatic generation from the CDI summary file created this morning with NEMO – Validation of the XML files
Creation of the corresponding coupling file

19. From ASCII data to XML files
VALIDATOR a b
Collection of
ASCII files
ODV
files
XML
CDI files
summary_CDI_NEMO.xml
Delivered with NEMO
Coupling file for DM
SeaDataNet
CDI
CDI summary
CSV file
Coupling table
MIKADO
Export
Coupling table

20. a) Configuration file for CDI XML
Run MIKADO
Open the configuration file delivered with NEMO :
MIKADO => Automatic/Open/CDI
the configuration file is in the HOME_DIRECTORY of NEMO, but for the training it is under \Practical work\config , file summary_CDI_NEMO.xml
Make the connection with the Excel file that you have created with NEMO (the Excel file must be closed to avoid conflict access)
Check the connection (you must have a green message)
Go to the Queries tag and have a look at the queries
Check them (green message after check all)
Save your configuration file

21. a) XML file generation
Run Automatic/Generate/CDI
Open the configuration file
\Practical work\config\summary_CDI_NEMO.xml
Choose the export directory (where CDI XML files will be written)
Choose the type of export file you want :
XML files
Zip file containing the XML files
Both (XML and Zip)
Open one XML export file to have a look at it with MIKADO Manual>Open>File>CDI

22. a) Web validation service: Validator
Check the XML file with the Web validation Service at :
Select the CDI catalogue
Validate file with CDI Xml schema
Upload one of the XML file that has been generated
Validate the file
Look at the result of the validation : you must have no error, you can have some warnings depending on the vocabulary versions.

23. b) Coupling configuration file
Run MIKADO
Open the configuration file delivered with NEMO :
MIKADO => Tools/Coupling table for download manager/ Import CDI configuration
the configuration file is the same than in a) ) in the config directory
Check the connection to the excel file
\Practical work\config\CDI_summary_with_address.xls
Go to the main query tag and check the query (you must have a green message)
Go to Modus 1 or Modus 3, choose the output format and fulfill the query which retrieve the file address in the excel file
Check it
Preview the coupling file
Save your coupling configuration file

24. b) Coupling file generation
Run the generation of the coupling file
MIKADO => Tools/Coupling table for download manager/ generate/ Modus 1 or Modus 3 depending on your ODV file(s)
Select the configuration file for coupling that you just create
Select the output directory
Choose the mode for writing
Generate the coupling file
Open it with a text editor

25. Exercise 2
EDIOS manual input of
1 Program
1 Series
1 Platform

26. Under SDN_tools\practical_work
Under \EDIOS__POSEIDON_Example\EDIOS_Data
Open the EDIOS_POSEIDON.xls file describing the EDIOS program, series and platforms (1 program, 15 series,15 platforms)
Input the program information into MIKADO and create the corresponding XML file
Input 1 series information into MIKADO and create the corresponding XML file
Input the platform related to the previous series in MIKADO and create the corresponding XML file
Why can’t you first enter a platform, then a series then a program?

27. Exercise 3 EDIOS Automatic generation of XML files for: Program Series
Platforms

28. Under SDN_tools\practical_work
\EDIOS__POSEIDON_Example\EDIOS_Data
Open the EDIOS_POSEIDON.xls file describing the EDIOS program, series and platforms (1 program,15 series, 15 platforms)
Run MIKADO to create the automatic query to retrieve the program information
Or open the already written XML configuration for MIKADO for EDIOS program : \EDIOS__POSEIDON_Example\queries\poseidon_edios_program.xml
Run MIKADO to create the automatic query to retrieve the series information
Or open the already written XML configuration for MIKADO for EDIOS program : \EDIOS__POSEIDON_Example\queries\poseidon_edios_series.xml
Or open the already written XML configuration for MIKADO for EDIOS platform : \EDIOS__POSEIDON_Example\queries\poseidon_edios_platform.xml

29. Generate the XML EDIOS files
EDIOS PROGRAM
Run MIKADO : Automatic>Generate>EDIOS Program
Choose the \queries\poseidon_edios_program.xml
Choose the export directory \results\program
EDIOS SERIES
Run MIKADO : Automatic>Generate>EDIOS Series
Choose the \queries\poseidon_edios_series.xml
Choose the export directory \results\series
EDIOS PLATFORMS
Run MIKADO : Automatic>Generate>EDIOS platforms
Choose the \queries\poseidon_edios_platforms.xml
Choose the export directory \results\platforms