1. Cognitive Service in AR platform.
Platform provides cognitive service that can be used by any application.
Uses Innovation Suite’s (Referred as IS) Cognitive service
Supports auto categorization today.
Given a text , classify/categorize into a category based on training provided
Key is training data!

2. High Level Architecture

3. Consuming Cognitive Capabilities - Steps.
Contact BMC to get IS service URL, username, and password.
Identify application form(s) that needs auto categorization capabilities
Identify Ifield(s) on the form that has description
Identify field(s) on the form that stores corresponding category
Identify existing data in this form that you would want to use for training the ML provider.
Generate training data & Train the service
Use the service to automatically categorize new data you would create in the form.

4. Configuration Setting Name Default Value Description
*Cognitive-Service-Url 
""
URL of the IS cognitive service
*Cognitive-Service-User-Name
Username provided by BMC to access IS cognitive service
*Cognitive-Service-User-Password
Password provided by BMC to access IS cogntive service
Cognitive-Service-Confidence-Threshold 90
AI providers typically return confidence when categorizing a given text. Server will not return suggested category if confidence level is below this threshold.
Enable-Cognitive-Service F
Enable/Disable Platform's Cognitive capabilities.

5. Data Set Descriptor form - AR System Administration:Cognitive Service Training Data Set Descriptor

6. Data Set Descriptor form
Entry in this form describes training data and how to generate the training data
Each entry has an ID – Data Set ID.
This is GUID – If not provided , server will generate one!
Training will be identified using this ID
Auto categorization of a text requires this ID – ML provider needs to know which training to use to categorize!
Why qualification and Training data generation algorithm?
You can’t send unlimited data for training –current limit is 15,000 rows
Qualification will identify the data
Algorithm will pick subset from the identified data.

7. Training Jobs form - AR System Administration:Cognitive Service Training Data Set Pending

8. Training Jobs form
Create an entry into this form in order to generate training data and train the service.
Data Set ID will be used to find data set descriptor entry that describes how to generate the training data.
Allows one to provide training data. If training data is provided in the entry (attachment) , server WILL just use that data to train the service.
Creating entry generates entry into Application pending form and server responsible for generating training data gets signaled. (See next slide)

9. Generating training data
Generating training data is server group operation!
Keeping future need in mind - Generating training data can become time and resource consuming.
Provides failover capability – other ranked server will pick up job of generating the training data should current server gets booted.
New server group operation “Cognitive Service” is introduced for this purpose.
Training data will be sent over to IS service which will in turn send the training data to ML Provider – Watson currently.
Training ML provider is done asynchronously as it can take time to train.
Ranked server will poll the status and update the status of the job accordingly.

10. Generating training data
Generating training data is server group operation!
Keeping future need in mind - Generating training data can become time and resource consuming.
Provides failover capability – other ranked server will pick up job of generating the training data should current server gets booted.
New server group operation “Cognitive Service” is introduced for this purpose.
Training data will be sent over to IS service which will in turn send the training data to ML Provider – Watson currently.
Training ML provider is done asynchronously as it can take time to train.
Ranked server will poll the status and update the status of the job accordingly.

11. Using the service - AR System Administration:Cognitive Service

12. Using the service
Service entry call on display only form - AR System Administration:Cognitive Service
Input
Command – categorize
50 fields for you to map text to classify
output
10 fields for returning category (Hierarchical categories)
Confidence field that ML provide returns
In case confidence provided by ML provider < configured threshold, category fields will not be returned.

13. Few limits
Training is not cumulative – you can not incrementally train.
As of now, you can only create 8 trainings.
Creating 9th training will give you entitlement error –you can always delete one or more of the 8 training and then create new once.
For training only 15,000 rows can be sent.
Only 1024 characters will be used from input text column of CSV file.

14. Troubleshooting
Log file arextension.log at <Server – db –directory>
Check the configuration – logs will indicate if server was able to connect to IS service or not.
Make sure service is enabled – again logs will indicate if not enabled.
Error text will say “IS Service” in case error returned by IS service.

15. Code New internal plugin is written
Internal plugin issues REST api calls to IS service
Nothing else was changed in server to deliver this feature except writing this new internal plugin.
More in code walkthrough.