The goal of BEARD is to systematically explore the adversarial robustness in Dataset Distillation (DD). A critical gap exists in the field, where the adversarial robustness of models trained on distilled datasets remains underexplored. Despite advancements in DD, deep neural networks continue to be vulnerable to adversarial attacks, posing significant security risks across various applications. Current research shows that while DD can partially improve adversarial robustness, it falls short of fully addressing these vulnerabilities. Additionally, the lack of a unified, open-source benchmark for evaluating adversarial robustness in DD has slowed progress in the field.

In response, we introduce BEARD, an open benchmark designed to systematically evaluate the adversarial robustness of DD methods. BEARD provides a comprehensive framework for testing DD methods across various datasets using different adversarial attacks, introduces new evaluation metrics, and offers open-source tools to support ongoing research and development.

TO DO List:

• Supplement More DD Methods: Our benchmark evaluates the following distillation methods: DC, DSA, DM, MTT, IDM, BACON, TESLA, RDED, and et cetera. We will periodically update this list as new methods are developed and evaluated.

  ✓ Completed methods
  ✗ Not yet completed methods

  We will periodically update this list as new methods are developed and evaluated.

• Improve the Code We welcome contributions and participation from the community to help enhance our benchmark. If you have suggestions or want to contribute, please feel free to get involved and make a difference!

Training phase with
6+ distillation methods
and 3+ datasets

Evaluation phase with
5+ adversarial attacks
to assess model robustness