❔ Understanding the difference between ML and AI security

[shsingh]

Is there a common set of guidelines and agreed taxonomy to define the difference between topics such as:

• Machine Learning
• Artificial Intelligence
• Deep Learning
• Reinforcement Learning
• Natural Language Processing

Some of these topics may be limited to the research/scientific realm and not for security practitioners.

❓ ## Questions for discussion

• What should be defined as part of the MLSEC TOP 10 to help people understand the differences?
• How do Large Language Models fit into above?
• Does a basic level of explanation need to be done for this project? Or can this be referenced elsewhere with another body of work or project/committee?

[robvanderveer]

Here's Rob from the AI guide.
Deep learning is a type of Machine learning is a type of AI.
Reinforcement learning is a type of machine learning in the sense that the model is trained by experimenting agents, instead of labeled data.
However, a system that employs reinforcement learning is by definition an optimisation system, to for example find the best path through a maze. In that sense it is different from supervised machine learning models -which try to predict, rate or classify.
Natural Language Processing is more of research area that deploys AI models.
Large Language Models are built on machine learning models, where the key type is to predict the next word based on the input and what the previously generated words are (simplified but basically).

ISO defines AI as
'set of methods or automated entities that together build, optimize and apply a model so that the system can, for a given set of predefined tasks, compute predictions , recommendations, or decisions'
with
'Predictions can refer to various kinds of data analysis or production (including translating text, creating synthetic images or diagnosing a previous power failure'

Therefore it rules out optimization systems that perform tasks like scheduling, planning . (Note that optimization as a mechanism is at the root of machine learning because the training process optimizes the model parameters to achieve the best performance).

The result is that AI under ISO consists of machine learning and so-called heuristic systems. ISO 5338 also discusses this. Heurstic systems that reason based on rules or patterns programmed by people instead of learning from data. They used to be called expert-systems, and currently they represent a relatively small part of the AI systems in production.

When it comes to model attacks, heuristic systems of course don't suffer from data poisoning per se, but they do suffer from evasion attacks and model theft.

So to answer your question: The MLSec Top 10 is by definition about machine learning which includes deep learning and generative AI models such as LLMs, and excludes reinforcement learning and heuristic systems. It may be good to cover some of these definitions in the guiding text with the Top 10, and explain that heuristic systems DO suffer from a few of the top 10 entries.

To wrap up, let's look at the AI act. It defines AI as 'software that is developed with one or more of the techniques and approaches listed in Annex I and can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environments they interact with.'

Annex I says:
(a)Machine learning approaches, including supervised, unsupervised and reinforcement learning, using a wide variety of methods including deep learning;

(b)Logic- and knowledge-based approaches, including knowledge representation, inductive (logic) programming, knowledge bases, inference and deductive engines, (symbolic) reasoning and expert systems;

(c)Statistical approaches, Bayesian estimation, search and optimization methods.

This is consistent with ISO, with the exception of the very last bit: 'Search and optimization methods'.

Search and optimization is typically an Operations Research (OR) problem, and certainly there is overlap with the field of AI.
I think the main question is: should optimization systems be part of AI discussions.
From an EU AI act perspective, I believe they should, as from personal experience I can confirm that optimization systems are dealing with fairness challenges (recently I helped to make sure that the Routescanner algorithm is fair towards ports worldwide).

From a security threat perspective I see optimization systems as a very different use case from machine learning and heuristic systems. It is better to leave them out of top 10 discussion to prevent create unnecessary noise, and cover them separately if there is a demand.

[edthedev]

Having just worked with a team working on an LLM - the difference we felt the most is that there's usually an un-trusted user interacting with an LLM - since the LLM is meant to process language and typically interacts directly with a user in a human language.

In contrast, ML efforts we have worked with are often further back in the software stack. The machine learning code may be quietly learning from data stores and making recommendations or taking automated actions - without as much direct human interaction.

A lot of the risks are the same, but there's enough difference that I currently expect my team to eventually maintain separate guides for LLM and ML vulnerability discussions.

[msnishanth9001]

Hi @robvanderveer that is a wonderful summary.

can you please clarify this.

The MLSec Top 10 is by definition about machine learning which includes deep learning and generative AI models such as LLMs, and excludes reinforcement learning and heuristic systems.

• But ML works with data and a cost function, to optimize on. It inherently does optimization, like any ML algorithm.
• I think it is not good to relate with supervised learning, as that works on a different optimization problem. ML models are classified based on the optimization problem.

Heuristic systems that reason based on rules or patterns programmed by people instead of learning from data.

• my understanding of Heuristic systems, perform an optimization on the rule imposed.
• ML algorithm try find patterns from the data based on the optimization problem. Here the cost function is imposed.
• In both cases there is learning from data based on the inputs.

To add, would it be apt to say

• DL (Deep Learning), Is to represent multiple layers in the ML architecture which can be any of the ML models. This does not advocate for the AI.
• LLM (Large Language Model) model is composed of multiple ML models (NLP, DL, RL) that collaborate in a structured manner to generate a targeted outcome.

I also agree to your AI definition from AI act as, software that is developed with ML models to complete a task with human precision or better.

[robvanderveer]

Sure. I will respond to your points gladly.
If you look at the AI guide threat model it shows actually two algorithms: the one that trains (optimizes) and then the one that performs that actual function, which is to apply the trained regularities to perform a task such as classification or prediction. The latter is the actual machine learning model. The functionality that matters is the classification/prediction, not the optimization.
And sure, the functionality determines the optimization problem, so ML models are classified on functionality and therefore also on the optimization problem.

Heuristics systems don't optimize, apart from finding the set of rules to fire (e.g. backtracking algorithm)- but they don't optimize the representation of knowledge, like ML models do.
Heuristic systems (as I and as ISO define them) do not learn from data.

DL refers to perceptron layers - not layers of various ML models.
RL is a training method to train LLM's, as opposed to supervised learning. It's not a type of model.
Neither is NLP. NLP is the type of application.

You paraphrase my discussion of the AI act definition of AI, where you state that AI consists of ML models. That is not what I wrote and also not what the AI act says. Note that the AI act explicitly includes heuristic systems (see my point b about Annex I).

[shsingh]

created wiki page for glossary: https://github.com/OWASP/www-project-machine-learning-security-top-10/wiki/Glossary

entries from wiki's glossary page will be added to the 'glossary' tab: https://owasp.org/www-project-machine-learning-security-top-10/

Is there a common set of guidelines and agreed taxonomy to define the difference between topics such as:

• Machine Learning
• Artificial Intelligence
• Deep Learning
• Reinforcement Learning
• Natural Language Processing

ref: #16