1. Tutorial of Enhanced CIMdesk Model Validation
Power Info LLC
November 2018

2. What is new in CIMdesk ?
In CIMdesk , the model validation function is significantly enhanced:
Adding many new validation rules based on
ENTSO-E CIMdesk-NMD Technical Requirements
“Quality of CGMES Datasets and Calculations (Third Edition)”
Supporting CGMES Validation: a new model validator designed to support and facilitate CGMES model exchange process

3. Purposes Purposes of CIMdesk 2.4.12 release:
Helping TSOs improve the quality of model submittal
Gathering feedback for the enhanced CIMdesk model validation function
Purposes of this tutorial:
Providing overview of new validation rules
Helping users learn how to use the new CGMES validator quickly

4. About New CIMdesk Validation Rules
New CIMdesk validation rules are added based on:
ENTSO-E CIMdesk-NMD Technical Requirements: A specification from ENTSO-E aimed to synchronize CIMdesk validation with Network Model Database (NMD) validation and enhance NMD validation.
“Quality of CGMES Datasets and Calculations (Third Edition)”: An ENTSO-E standard aiming to increase the quality and consistency of CGMES data that is exchanged to support various business processes.
Currently, about 95% of the new rules are implemented in CIMdesk The full set of new rules will be implemented in CIMdesk 2.5.

5. A Sample Validation Rule from ENTSO-E CIMdesk-NMD Technical Requirements

6. A Sample Validation Rule from “Quality of CGMES Datasets and Calculations (Third Edition)”

7. What will be the impact of the new validation rules on TSO model submittal?
Most likely, your model submittal which passes CIMdesk 2.4 validation will fail to pass validation in CIMdesk You will see many errors in the validation report.

8. What cause these errors?
The new validation rules enforce many model quality improvement rules. For example, Level 6 validation of “Quality” ensure your power flow settings is based on “a solved case”.
The new validation rules are quite restrictive. Some of them are debatable. Many of them are not tested with reality-based models.

9. How to eliminate these errors?
Perform an in-depth analysis for each reported error (and warning), and
If it is your data issue, fix it or contact your vendors to have them fix it;
If the new rule that triggers the error doesn’t make sense to you, contact ENTSO-E and have the rule removed or modified.

10. So, what is really behind CIMdesk validation?
CGMES Profile
Quality of CGMES Dataset
CIMdesk/NMD Requirements
CIMSpy Validation
Network Topology Analysis
XML/RDF/UTC/UUID/… Syntax Checker

11. About CGMES Validator
CGMES Validator is a new CIMdesk model validation function specifically designed to support CGMES model exchange process where self-descriptive TSO model parts are exchanged individually.
The existing profile-based model validator is still supported.
The remaining portion of the tutorial works best while performing some hands-on during the self-learning process.

12. Terminology – Model Part
In CGMES, models are split into different data group (EQ, SSH, TP, SV…), referred to as model parts. Model parts are self-descriptive. Each model part contains a header which contains the meta-data describing the model authority, profile conformed, scenario time, and dependency on other model parts.

13. Terminology – Boundary Set (BS)
Boundary Set (BS) defines the tie-connections among TSOs. BS is prepared and maintained by ENTSO-E.

14. Terminology – Individual Grid Model (IGM)
Individual Grid Model (IGM) is a collection of related model parts prepared by each Model Authority (MA), i.e. TSO in the context of CGMES.

15. Terminology – Extended Individual Grid Model (EIGM)
Extended Individual Grid Model (EIGM) is a variation of IGM. An EIGM consists of a single IGM and the BS it depends on.
IGM is not a complete data set in terms of power flow study. EIGM, however, is a complete dataset, supporting full validation of IGM.

16. Terminology – Common Grid Model (CGM)
Common Grid Model (CGM) is made up of interconnected BS and IGMs representing a specified scenario. CGM is a complete data set. It is used to calculate a power flow solution of the entire region for a specified time or time duration. CGM can exist in different formats:
BS + IGMs: Each data set, in its original format, corresponds to the same (or the nearest) scenario.
BS + IGMs + Shared CGM Model Parts: The results of the CGM power flow study (i.e. a single SV model part) are combined with the input, creating a simulation snapshot of CGM for a time horizon.
Assembled: Model parts of CGM in the above formats are assembled as a single and complete model.

17. Terminology - Workspace
A workspace is in-memory model depository where different kinds of model parts can be loaded. CIMdesk relies on its model discovery algorithm to group model parts into BS/IGM/CGM based on model dependency description of the loaded model parts. In workspace, model parts are grouped based on the following rules:
Priority 1: CGM Formation – creating CGMs if discovered. A CGM contains a BS and at least 2 IGMs. A CGM is uniquely identified by an EQ_BD.
Priority 2: Extended IGM (EIGM) Formation – creating a model group consisting of a single IGM and the BS it depends on. The reason of formation of EIGM is to facilitate IGM validation.
Priority 3: IGM Formation – Model parts are grouped as IGMs if they don’t belong to any of grouped CGMs or EIGMs. An IGM is uniquely identified by an EQ.
Priority 4: BS Formation – if no IGM model parts depend on the loaded BS model parts, the loaded model parts will be grouped as standalone BS.
Priority 5: Marking of Ungrouped Model Parts – Model parts will be marked as ungrouped if they don’t belong to any of the above.

18. Why is a new validator needed?
Because the existing CIMdesk profile-based model validation is inconvenient and inadequate to support CGMES model exchange process.
In order to validate the model (IGM/BS/CGM) comprehensively, model parts must be
Validated individually, and
Assembled, and
validated together as an assembled model
To identify all of the modeling issues, users much check validation reports generated during model part validation, model assembling, and assembled model validation, and sort through the generated validation results with duplicity and overlapping.
Because different modeling practices are permitted in CGMES, the assembled CGM could be a mixed node/breaker and bus/branch model. Existing profile-based model validation assumes either node/breaker or bus/branch, but not both. As a result, false alarms will be generated for the bus/branch portion of the assembled model.

19. Design Goal of CGMES Validation
CGMES model validation is an automation process that
Streamlines three validation steps and provides one-step solution for comprehensive model validation in a broader context ;
Improves the usability of model validation by facilitating analysis of model validation results.

20. How does it work?
During CGMES validation, model parts are validated individually AND together with other related model parts.
The reported validation incidents are a union of the three validation processes (i.e. model part validation, model assembling, and assembled model validation)with redundant validation incidents eliminated.
While the model parts are assembled to support model validation in a broader context, the process is automated and end users don’t even see the intermediate assembled models. Additionally, the validation incidents are directly related to the related model parts.

21. A First Look at CGMES Validation Report

22. How to Run CGMES Validation (1)?
From context menu associated with the selected model part

23. How to Run CGMES Validation (2)?
From the dropdown command icon at the header of the summary display of the selected model part.

24. How to Run CGMES Validation (3)?
Workspace validation can be invoked from the global context menu.

25. What is behind CGMES Validation?
Quite complicated. If you are really curious, check User’s Guide for the detailed logics behind CGMES validation.

26. About IGM Validation
It is designed mainly for TSOs to validate their IGM submittal.
ALWAYS run IGM validation with BS it depends on (i.e. EIGM validation), because IGM itself is not a complete data set.

27. About CGM Validation
It is designed mainly for ENTSO-E to validate collected IGMs (of a time horizon) as a whole.
Whenever possible, TSOs should also run CGM validation to check balancing at boundary points with neighboring TSOs.
CGM validation can only be enabled with at least two IGMs loaded along with the shared BS.

28. About Workspace Validation
Workspace validation is a free-style validation based on the built-in data grouping rules and heavily relying on the meta-data in the header of model parts.
The fundamental principle behind Workspace validation: whenever possible, model parts shall be validated in a broader context.
Workspace validation can always be invoked, enabling users to identify issues related to missing or misleading model dependencies. Such issues could result in disabling of other types of CGMES validation.

29. What is the difference between Workspace validation and assembled model validation?
Majority of the reported incidents are identical for two validations.
Some of the incidents may not be able to discovered after model parts are assembled.
Workspace validation is more comprehensive. It reports:
all of the incidents discovered during parsing of model part files;
all of the incidents when each model part is validated individually;
all of the incidents discovered during consistency checking between model parts;
all of the incidents that IGM validation fails to report due to missing/misleading model dependency descriptions;
all of the incidents when model parts are assembled;
all of the incidents after model parts are assembled and validated as a whole.
The results of the assembled model validation are easier to inspect.

30. How to validate assembled CGM?
Assembled CGM could be a mixed node/breaker and bus/branch model, part of which could support ShortCircuit.
DO NOT use Workspace validation for assembled CGM, since the header of the assembled CGM doesn’t apply to all IGM parts assembled.
Assembled CGM should be validated against base profile.
If you want to validate assembled CGM ‘Operation’ sub-profile, the assembled CGM shall be converted to a full node/breaker as described later.

31. What is the difference between “Full BS/IGM/CGM” and “as Part of BS/IGM/CGM”?
Fundamentally the same, but showing different validation results:
“Full” validation displays all of the incidents for all of the model parts;
“as Part” validation displays only the incidents related to the selected model part;
“Full” validation can only be invoked from the ‘master’ model part (for example, EQ for IGM, EQ_BD for CGM), while “as Part” validation can be invoked from any model parts.

32. Why are some expected menu items disabled and some of the expected command icons missing in the dropdown?
This typically means either related model parts are missing or the related model dependences are absent or incorrect.
CIMdesk has a built-in intelligence to determine what types of CGMES validation should be supported for each loaded model part.
Run Workspace validation to find out what is wrong.

33. How to pinpoint issues from validation reports?
In model validation reports, the reported incidents normally have a navigation link to the related objects in the related model part.
In some cases, the reported issues are in other model parts. This is because there are no related objects in the model part where the issue occurs. In the following example, the reported incident has a navigation link to object in the EQ model part, but the issue is actually in the SV model part. There is no way to report this issue in the SV model part.

34. Tips for Model Debugging
Problems:
Because CGMES model validation checks consistency between model parts, understanding of some of the reported incidents requires navigation between model parts. It is quite inconvenient.
Diagrams help users understand the reported network connectivity/topology issues. However, some diagrams may not be available unless the model parts are fully assembled.
Solutions:
Most issues reported during CGMES validation will appear in the assembled model. It is suggested that users assemble the model parts and run the profile-based validation to facilitate interpretation of some of the incidents reported in CGMES validation.

35. CGMES Validator doesn’t work as expected for my IGM. What should I do?
Check the header of your model parts. CGMES validation heavily depends on meta-data description in the header of model parts.
It uses model dependency description to determine what types of CGMES validation shall be enabled and how model parts shall be assembled and validated together as a whole.
It uses your profile URIs to determine what versions of CGMES profile shall be used for validation.
It checks the consistency between model parts using information in the header, such as scenarioTime.
It relies on the sub-profile URIs to determine what types of sub-profiles (i.e. Base, Operation, or ShortCircuit) shall be used for validation. …

36. Enabling Model Authority Related Validation (1)
Problem: In CIMdesk, some of the validation rules require users to specify the corresponding model authority for the specified IGM. These rules are used to validate the boundary connections However the model authority set (MAS) declared in the model header is not standardized. These rules cannot be validated.
Solution: In order to enable model authority related validation rules, users must explicitly declare the model authority for the imported IGM. This is accomplished by “SetModel Authority” for the EQ model part. Once the model authority is selected, the related validation rules will be enabled. Users must re-run the validation.

37. Enabling Model Authority Related Validation (2)
NOTE:
The model authority setting is not stored. That means users will have to set the model authority each time after a new IGM is loaded.
If IGM is assembled, users must also set the model authority for the assembled model.
This feature is intended to be a temporary solution. Once MAS is standardized, model authority setting will no long be needed.

38. Operational Model vs. Planning Model (1)
In CGMES, there are two types of models: Operational Model and Planning Model. The former is typically the reality-based as-built model, while the latter is for future planning, such as ENTSO-E Ten-Year Network Development Planning (TYNDP).
While the two kinds of models follow mostly the same CGMES modeling practices, there are some differences regarding model validation:
Planning Models use some conventions, for example, using attribute IdentifiedObject.description to carry some additional information, such as fuel type category, PINT flag, etc.
Planning model vendors are not fully ready to support CGMES Quality of Data Set and Calculation. Therefore, some of the validation rules are excluded, and some of the errors are downgraded to warnings.
Planning Models need to be validated against the future boundary set and Pan-European Market Modelling Data Base (PEMMDB).

39. Operational Model vs. Planning Model (2)
CIMdesk enables users to switch between Operational and planning modeling environments, supporting validation of both types of models.
It is important to note that the terminologies of Operation and Planning are different form those defined in CGMES profiles where Operation means node/breaker and Planning refers to bus/branch.
Modeling environment is per Workspace, thus multiple modeling environments can co-exist.
The default modeling environment is ‘Operational’, but it can be changed to ‘Planning’ through configuration.

40. PEMMDB-Related Validation
In Planning Modeling Environment, CGMES validator checks the loaded IGM against the dataset from Pan-European Market Modelling Data Base (PEMMDB).
By default, the PEMMDB dataset is stored as a configuration file (i.e. config\custom\cimdesk\PEMMDB.csv).
Users can change the PEMMDB dataset at run time. To change the PEMMDB dataset, the workspace must be in the Planning Modeling environment. The PEMMDB dataset is a global setting. It is applicable to all of the models loaded in.
The PEMMDB-related incidents are displayed as a new category, like Topology-related incidents. By default, PEMMDB-related incidents are not displayed. Users can display such incidents by clicking the corresponding icon.

41. Additional Meta-Data Required by CGMES Validation (1)
NMD model validation requires some additional meta-data to be provided.
Some of the meta-data files are overwritten by those passed from NMD host application, while others are treated as part of CIMdesk configuration files.
These meta-data files, located under folder config\custom\cimdesk, are summarized below:

42. Additional Meta-Data Required by CGMES Validation (2)
File Name
File Type
Description
CIMdeskConfig.xml
XML
This is the master configuration file for CIMdesk related configurations, including definition of different tolerances, file paths of other configuration files, etc.
FuelTypes.csv
CSV
This file defines ENTSO-E fuel types, including category, fuel, type, class, etc. It is used to drive fuel-type-related validation and reporting. This file doesn’t change frequently. It is treated as part of CIMdesk configuration file.
ModelAuthorities.csv
This file defines TSO model authority short names. These short names must match those defined in the NMD boundary set, i.e. fromEndNameTso and
toEndNameTsocategory of the boundary TopologicalNodes and boundary ConnectivityNodes. When validating an IGM, the model authority short name passed from NMD must also match one of those defined in this file. This file doesn’t change frequently. It is treated as part of CIMdesk configuration file.
PEMMDB.xml
Pan-European Market Modelling Data Base (PEMMDB) defines the market data for a specific time scenario or time horizon, such as TSO net interchange, generation capacity/output, and demand, etc. The PEMMDB data are normally passed from NMD at run-time. The default PEMMDB.xml is normally for testing or standard-alone CIMdesk.

43. How can I ensure that my IGM submittal passes NMD validation?
Set the modeling environment as ‘Planning’;
Load your IGM and the Boundary Set (BS), making sure that your BS is the same as the BS from NMD and for the targeted time scenario;
Set the model authority for EQ model part;
Run Workspace validation;
No Errors shall be reported.

44. My IGM passed CGMES Validation in CIMdesk
My IGM passed CGMES Validation in CIMdesk. But when it was submitted to ENTSO-E NMD, it failed to pass the NMD validation, consequently rejected. What should I do?
Make sure that you are running the latest version of CIMdesk.
NMD perform more in-depth validation using some data from external data sources, such as PEMMDB. You may not have such data or the data may not be properly updated.
Always update your boundary sets to be in-sync with the BS used in NMD validation, particularly if you see validation error like this from NMD.

45. For more information about CGMES validation, please refer to the corresponding article in User’s Guide.

46. CGMES Reporting
A variety of built-in CGMES reports are provided in CIMdesk
The reports are supplemental to CGMES validation.
While the CGMES reports are available for any models with EQ components, these (full) reports shall be called from the assembled model.
All of the reports can be exported as CSV files through context menu ExportCGMESAll of Above

47. Alternative Way to Call CGMES Reports
Like other custom reports, CGMES Reports can also be called from the dropdown icons at the header of Summary Display of the model.

48. Other Enhancements in CIMdesk 2.4.12
Added support for multiple workspaces, a featured designed to support multi-tasking in different modeling environments ;
Added a new feature to convert Bus/Brach to pseudo Node/Breaker;
Enhanced model assembling report;
Fixed a dozen of SPRs since release of CIMdesk 2.4.

49. Supporting Multi-Workspaces
The concept of workspace is introduced mainly to support multi-tasking. It enables users to perform multiple modeling tasks concurrently and independent of each other. A workspace is an independent modeling environment containing:
In-memory data repository: storing the modeling information that users are currently working on.
Associated modeling applications: modeling applications, (such as model validation, model assembling, etc.,) that users can invoke to perform the corresponding modeling tasks in the workspace.
Configuration: special configurations applicable to the workspace, mainly to customize the modeling environment.

50. Workspace Assembling
Automatic model assembling heavily relies on complete and accurate descriptions of model dependencies at the header of the models.
In some cases, this pre-condition doesn’t hold, particularly between IGM and Boundary Set.
Workspace assembling is a free-style model assembling process during which model parts loaded in the workspace are assembled in a pre-defined order (i.e. Boundary EQ, Boundary TP, EQ, SSH, TP, SV…). Model dependencies are ignored during workspace assembling.

51. Conversion of Bus/Branch to Pseudo Node/Breaker(1)
In CGMES, different modeling practices may be used to build and maintain IGMs. For example some of IGMs are node/breaker models, while others are bus/branch models. Both are accepted modeling practices according to CGMES. Consequently, the resulting assembled pan-European is a mixed bus/branch and node/breaker model. Such an assembled model may be difficult to consume. This is particularly true if the model consumption applications/systems deal with only node/breaker model.
To facilitate model consumption of assembled model, CIMdesk provides a new data engineering function to create pseudo node/breaker from bus/branch. This allows users to convert a bus/branch model to a node/breaker model, or a mixed bus/branch and node/breaker to an assembled model supporting both (full) node/breaker and bus/branch.

52. Conversion of Bus/Branch to Pseudo Node/Breaker (2)
This function can be invoked from the context menu ProcessTo Node/Breaker.
The function is only available for the assemble model which contains at least EQ and TP.

53. Conversion of Bus/Branch to Pseudo Node/Breaker (3)
The resulting is an assembled model that fully supports both bus/branch and node/breaker.
rdf:IDs of the added ConnectivityNodes can be kept in-sync with the corresponding TopologicalNodes via configuration.

54. Enhancement of Model Assembling Reports
Model assembling reporting has been enhanced in CIMdesk to support master/details reports, facilitating inspection of issues occurred during model assembling.

55. Most-Wanted: Feedback
One of the major reasons for CIMdesk release is to gather feedback from TSO end users so that the new features implemented can be further improved and the new validation rules added can be tested and agreed upon within the ENTSO-E communities. The goal is to improve the new features and new rules in NMD and CIMdesk 2.5.
For issues related to new features or improperly implemented validation rules, please contact and CC ENTSOE-E secretariats.
If you want to challenge a new validation rule, please contact ENTSOE-E secretariats and CC