Garak (software)
garak is a computer security tool that provides information about LLM security vulnerabilities and aids in penetration testing and red teaming of language models and dialog systems. It is supported by Nvidia. Officially the name is short for "generative AI red-teaming & assessment kit". garak is described as the leading LLM vulnerability scanner in an independent 2024 review by Fujitsu Research.[1] It is used and recommended as tooling in articles from Microsoft,[2] Trend Micro,[3] NVIDIA[4] and Cisco,[5] and has been covered in major IT news outlets.[6][7] Historygarak was developed in Spring 2023 by Prof. Leon Derczynski of ITU Copenhagen[8] during a sabbatical at the University of Washington. It was first released under GPL on 13 June 2023.[9] The license was later updated to Apache 2.0. The software is now homed at NVIDIA, where it lives as an open-source project with long-term support, and has been available via the NVIDIA public GitHub since November 2024.[10] FrameworkThe main components in garak are probes, generators, and detectors.[11] Probes manage attacks and implement an adversarial technique. Generators abstract away targets, which may be an LLM, a dialogue system, or anything that can take text and return text (plus optionally other modalities). Probes attempt to attack generators and pass the resulting output to a detector. The detectors assess whether or not the output indicates a successful attack. The whole is compiled into reporting by an HTML page and a JSON object summarizing results. See alsoReferences
External links
|
||||||||||||||||||||||||
