Adversarial Robustness and Security Risks in Machine Learning and Large Language Model Systems

Abstract

Machine learning systems, and more recently large language models, are now used in access control, financial decisions, clinical triage, navigation, and general-purpose agents. Many of the failures that matter are not ordinary implementation defects. They come from statistical decision boundaries, from training data, and from the natural-language channel used to operate the model. This survey organizes that risk as one lifecycle. It covers classical adversarial machine learning (evasion, poisoning, extraction, and inference attacks on discriminative models) and the surfaces that arrived with large language models: prompt injection, jailbreaking, training-time backdoors, insecure handling of model output, exfiltration, training-data extraction, multimodal injection, and agent-memory poisoning. We review representative techniques, from projected gradient perturbations and transferable triggers to optimized adversarial suffixes, query-only jailbreak search, many-shot prompting, and indirect injection through retrieval. Defenses are compared on the guarantee they actually offer: adversarial training, certified robustness, alignment by human feedback and written principles, injection defenses from instruction hierarchy to privilege-restricted architectures, runtime guardrails, red-teaming, and differentially private training. Evaluation practice, regulatory context, and composite incident patterns are treated as part of the same problem. The open issues are the lack of general certificates for language models, the tension between capability and controllability, the absence of a standard adaptive evaluation, and compositional risk in multi-agent systems. The aim is a reference for reasoning about security across the whole life of an AI system, including the parts outside the model.