1. Cybersecurity and artificial intelligence

2. Cybersecurity and artificial intelligence
The European Commission’s science and knowledge service
Joint Research Centre
Cybersecurity and artificial intelligence
Emerging trends
Dr. Ignacio Sanchez

3. The Joint Research Centre (JRC) at a glance
JRC's Mission
"As the science and knowledge service of the Commission our mission is to support EU policies with independent evidence throughout the whole policy cycle"
3000 staff
Almost 75% are scientists and researchers.
Headquarters in Brussels and research facilities
located in 5 Member States.

4. Cyber & Digital Citizens’ Security Unit Mission
To strengthen trust and security of the European Citizen in a sustainable and inclusive ICT-based European society by scientific research on how emerging Information and Communication Technologies will impact on the security and privacy of citizens’ daily life.
To work on risk mitigation, on cybersecurity, cybercrime, data protection, privacy and on the associated legal and regulatory frameworks aiming at a balance between European security needs and fundamental citizen rights including from the perspective of the emerging Digital Single Market.

5. European Commission's Joint Research Centre (JRC) flagship report on Artificial Intelligence
Joint Research Centre’s report on Artificial Intelligence
(5 December 2018)
ISBN: (online), (print),
ISSN: (online), (print),
DOI: /11251 (online); / (print); /91283
URL:
Coordinated Plan on Artificial Intelligence
(7 December 2018)
Communication from the Commission to the European Parliament, the European Council, the Council, the European Economic and Social Committee and the Committee of the Regions - Coordinated Plan on Artificial Intelligence (COM(2018) 795 final)
Communication Artificial Intelligence for Europe
(25 April 2018)
Communication from the Commission to the European Parliament, the European Council, the Council, the European Economic and Social Committee and the Committee of the Regions on Artificial Intelligence for Europe.

6. Artificial Intelligence
“Artificial intelligence (AI) refers to systems designed by humans that, given a complex goal, act in the physical or digital world by perceiving their environment, interpreting the collected structured or unstructured data, reasoning on the knowledge derived from this data and deciding the best action(s) to take (according to pre-defined parameters) to achieve the given goal. AI systems can also be designed to learn to adapt their behaviour by analysing how the environment is affected by their previous actions.
As a scientific discipline, AI includes several approaches and techniques, such as machine learning (of which deep learning and reinforcement learning are specific examples), machine reasoning (which includes planning, scheduling, knowledge representation and reasoning, search, and optimization), and robotics (which includes control, perception, sensors and actuators, as well as the integration of all other techniques into cyber-physical systems).”
European Commission's High-level expert group on Artificial Intelligence. "A definition of AI: main capabilities and scientific disciplines", 18 December 2018

7. AI Machine learning A new paradigm What has changed?
More computing power
More data
New algorithms
When using machine learning, human developers no longer program an algorithm to tell the computer how to solve a given problem but instead they program it to teach the computer to learn how to solve the problem.

8. Cybersecurity and AI & DT: 4 angles of influence
Opportunities
Challenges
Resilience
1. AI to create smarter cybersecurity
E.g. more effective security controls (antivirus, intrusion detection and prevention, scanning), automatic discovery of vulnerabilities, automatic malware analysis, smarter cyber defence, etc.
2. Robustness/Vulnerability of AI algorithms
E.g. adversarial machine learning (i.e. exploitation of weaknesses in AI algorithms to change their behavior), attacks against AI powered cyber-physical systems, etc.
Deterrence
4. Use of AI to fight cyber attackers & criminals
E.g. better biometrics, smarter forensics, fraud analysis, encryption challenge, fight against fake news, etc.
3. Misuse of AI
E.g. creation of deep audio video fakes, AI powered malware, smarter social engineering attacks, etc.  4 1 3 2
Conceptual model depicting the logical links between the different components of the cybersecurity risk in the context of the influence of AI and Digital Transformation

9. AI to create smarter cybersecurity
ANGLE 1 | Smarter Cybersecurity
Smarter cybersecurity controls
Automatization in labour intensive tasks
Better biometric controls
Network Intrusion Detection and Prevention systems
Malware detection
filters (spam, phishing, etc.)
Automatic discovery of vulnerabilities
Automatic exploitation of vulnerabilities
AI applied to malware analysis
Security logs and event correlation
Automatization of security operations (e.g. incident response)
Cybersecurity exercises (attack/defense) and training
Awareness raising

10. Autonomous security operations
ANGLE 1 Example | Smarter Cybersecurity
2016 Cyber Grand Challenge Final Event
Real time visualisation of a buffer overflow exploit in CGC
DARPA Cyber Grand Challenge (CGC)
URL:

11. Machine learning: a new paradigm
ANGLE 2 | Robustness/Vulnerability of AI algorithms
AI cannot be made unconditionally safe, like any other technology.
AI software based on machine learning techniques is found to not be robust against malicious attacks that exploit a specific set of AI vulnerabilities.
Thus: AI itself will likely be exploited and misused, for example by criminals.
Of highest importance: securing cyber-physical systems employing AI.

12. Classified as: ostrich.
Adversarial Machine Learning
ANGLE 2 Example | Robustness/Vulnerability of AI algorithms
Example of adversarial Machine Learning in image recognition
Classified as: dog.
Add crafted noise.
Classified as: ostrich.
Fooling a state-of-the-art object recognition neural network through the simple act of
Introducing specially crafted noise (Szegedy et al. 2013)

13. Adversarial Machine Learning
ANGLE 2 Example| Robustness/Vulnerability of AI algorithms
Left: real graffiti on STOP sign.
Right: designed stickers to mimic the situation and fool an object recognition AI.
>= 83 % success rate in fooling a state-of-the-art object recognition system in real drive-by tests
“Robust Physical World Attacks on Deep Learning Models” (Eykholt et al. 2018)
Example of a real life adversarial attacks

14. Influence of AI in cybersecurity threats
ANGLE 3 | Misuse of AI
New type of cyberattacks
Automatization of labour
Usage of AI for the evasion of cybersecurity controls (e.g. captcha solvers, antispam/antiphising filters, malware detection)
Autonomous / automatic decision making in cyberattacks (e.g. autonomous lateral movements in cyberattacks)
Enhanced social engineering and identity theft (i.e. realistic imitation of drafting style, voices and video)
Abuse of AI limitations
Some cybercrime labour intensive tasks can be automated, enabling for example:
Massive spear phishing attacks (highly targeted)
Highly interactive and scalable social engineering attacks (e.g. chat bots over IM, or social media)
Automatic botnet management
Faster, more interactive and scalable attacks
Automatization of cybercrime processes (e.g. ransomware payments)
More interactive
More effective
Cheaper
More targeted
Faster
AI can make cyberattacks …
More autonomous
Scalable
More difficult to attribute

15. AI-enhanced cyberattacks
ANGLE 3 Examples | Misuse of AI
AI-based attacks against authentication systems
Social Engineering, deception and Identity theft
Captchas
Identity theft and social engineering
Deep video fakes
Fake audio voice
Fake news
Phishing and cyber extortion
More targeted (e.g. impersonation of friends; location of targets in social media)
More contextualized (e.g. language translation, use of personal information)
AI applied to malware
Autonomous decision making in malware (lower dependence on remote control)
Evasion of malware detection controls
Self-maintained, self-protecting and evolving botnets
Automated interactive attacks
Massive spear phishing
Automatic ransom negotiation and assistance
Brundage, Miles, et al., "The malicious use of artificial intelligence: forecasting, prevention, and mitigation”, arXiv preprint arXiv:  (2018)
Sivakorn, Suphannee, Jason Polakis, and Angelos D. Keromytis, "I’m not a human: Breaking the Google reCAPTCHA”, Black Hat (2016)

16. AI in Law Enforcement ANGLE 4 | AI to fight (cyber)crime
AI can help in:
Reinforcing investigative capabilities
Strengthen digital evidence-making in court
New AI based tools:
Biometrics
Image / audio / video analytics
Localisation fingerprinting
Evidence identification and correlation
Encryption challenge
Automatic scanning
Digital forensics (data in transit, data at rest)
Analysis of local and global context
WHO
WHERE
WHAT
AI will help not only in the fight against cybercrime but also cyber-dependant crime and more generally crime

17. Visual Tattoo Inspection
Automatic Tattoo recognition
ANGLE 4 Example | AI to fight (cyber)crime
From classical text-based search (keywords) …
1st candidate
2nd
3rd
Visual Tattoo Inspection
Keyword(s)
Keyword DB
Matches
HUMAN/SKULL,
OBJECT/FIRE,
OBJECT/WEAPON
Human image interpretation is subjective
The process is tedious and therefore error-prone
The precision of recognition depends on the tattoo description
1st candidate
2nd
3rd
Tattoo Identification
To JRC ML / Deep Learning feature-based approach …
Deep Learning Model
Deep Learning Model
Deep Learning or SVM
Tattoo Detection
Segmentation
Learnt features
Tattoo
Description
Real datasets used: FBI Tattoo Database (NIST/Tatt-C)
JRC's research on Automatic Tattoo recognition

18. Smarter image /audio/video forensics
ANGLE 4 Example | Camera identification
SPN extraction
Camera fingerprinting
From picture to video
JRC's research on smarter image / audio / video forensics

19. Stay in touch EU Science Hub: ec.europa.eu/jrc Twitter: @EU_ScienceHub
Facebook: EU Science Hub - Joint Research Centre
LinkedIn: Joint Research Centre
YouTube: EU Science Hub