An Adversarial Benchmark for Firearm Detection in 3D Printing
3D printers can now produce functional firearms parts directly from a digital model, with no specialized skill or equipment. These "ghost guns" are cheap, unregistered, untraceable, and often invisible to the metal detectors in schools, theaters, and airports. Seizures of 3D-printed firearms by the NYPD rose from about 100 in 2019 to over 600 in 2022, reaching roughly 10% of all weapons seized in 2024, and they are now considered one of the fastest-growing threats to public safety.
States like New York and others are passing legislation requiring firearm-blocking technology at different stages of the 3D-printing pipeline. At the same time, private institutions like schools, universities, or individual manufacturers are considering implementing such measures on their own.
These decisions are made by people with limited technical background, who may reasonably worry: do we actually have the technology to enforce this? The purpose of this public code repository is to answer that question. While no detector is perfect, and there will always be room for research on more robust and efficient techniques, the technology is — and indeed has long been — here.
Caveats and assumptions
Before we jump into the conclusions, we must acknowledge some assumptions that this simplified project is making:
- One class of threat. Every "malign" file here is a Machine-gun Conversion Device (MCD), the small part that converts a semi-automatic handgun into a fully automatic weapon. "Benign" files are ordinary 3D models (everyday scanned objects and mechanical CAD parts). The detectors decide "is this one of our known MCDs?", not "is this any kind of weapon part?"
- We assume access to the complete 3D model. A printable firearm travels from a 3D model file
(
.obj/.stl) to slicer software, to machine instructions (.gcode), to the printer, to a physical object. This process reveals many intervention points: in this repository, we consider the easiest: the slicer stage, where one has access to the complete 3D model. - We recognize known designs. The detectors are trained on a catalog of known device designs and recognize those designs and disguised versions of them. A genuinely novel design that is not in the catalog will go undetected.
How this benchmark works
This benchmark has three components: a dataset, simulated adversaries, and simulated detectors.
An adversary is an agent that tries to evade detection by modifying a device's file while keeping it printable — for example, by re-posing it (rotation and translation), adding small random noise to its surface, completely re-meshing it (rebuilding the surface from scratch so the underlying file is entirely different), reordering the data inside the file (no change to the shape, but a different digital fingerprint), appending broken or invalid geometry meant to crash a careless checker.
A detector is a method that looks at a (possibly adversary-modified) geometry and decides whether it is one of the Glock MCDs in the dataset or not. We implement four key detectors:
- File hashing — It computes an exact digital fingerprint of the file and matches it against a list of known device files. It catches only byte-for-byte copies; any edit, even one that does not change the shape, defeats it.
- Geometric signature — a classical, non-AI method. It computes an intrinsic "shape fingerprint" from how the surface would naturally vibrate (its Laplace–Beltrami spectrum, like the distinct overtones of a bell), a quantity that does not change when the object is rotated, moved, rescaled, or re-meshed. It compares each connected piece of a model against the known-device fingerprints.
- AI-driven 3D classifier — a neural network (PointNet) that samples points across the surface and learns to tell device from non-device directly from the resulting point cloud, trained to ignore orientation.
- AI-driven 2D classifier — an image-based neural network (a multi-view CNN) that renders the object from many camera angles and runs the images through the same family of model used for ordinary photo classification, combining the views into one decision.
Finally, the dataset is private and contains, for now, 28 Glock MCDs, each in the form it comes in at the slicer. After applying all of the adversaries' modifications, this results in about 9,200 malign shapes and 4,400 benign shapes. As is traditional in benchmarks like these, we split the data into training data (which the models may look at beforehand) and held-out evaluation data — 1,000 benign and roughly 4,000 malign shapes, about 5,000 shapes the models never see. Every metric we report is computed on that held-out set.
Results
| Detector | Approach | Accuracy (AUC) | Time / file |
|---|---|---|---|
| Geometric signature | geometry / math | 1.0000 | 107 ms |
| AI-driven 2D classifier | AI (images) | 0.9989 | 119 ms |
| AI-driven 3D classifier | AI (point cloud) | 0.9938 | 46 ms |
| File hashing | industry status quo | 0.50 | 1 ms |
The accuracy reported above is the "area under the curve", the traditional metric for evaluating binary classifiers: 1.0 is perfect, while 0.5 is as good as a coin flip. The geometric signature is essentially perfect: a 1.0000 AUC (0.99999809), raising zero false alarms across all 1,000 benign models and missing only 2 of the ~4,000 evaluation devices. Crucially, no single adversary manages to pull any of the three real detectors below ~99% accuracy. And detection is fast: from about 1 millisecond per file for hashing to roughly 50–120 milliseconds for the geometry and AI methods — comfortably within the budget of a model upload or a slicing step, so screening adds no perceptible delay.
Conclusion
The technology is there. Even the most basic baselines (a decades-old geometric signature and standard off-the-shelf neural networks that are well obsolete by now) reach near-perfect accuracy, and they do so on a small dataset, under active disguise. Every method here was trained and evaluated on a single MacBook Air, with a fraction of the data and compute available to an industry lab, a government, or even most volunteer-run open-source projects.
If a working detector is this reachable (on a laptop, by a small academic team, with limited data) then the technology to underpin these emerging policies is well within reach. The open question is no longer whether it can be done, but how to standardize, govern, and scale it responsibly.
Limitations
This shows the technology is there; of course, this is a prototype, and it still lacks important components one would need before deployment:
- Glock MCDs only. This works on Glock MCDs because they are the most represented class in our private dataset. The methods themselves are generic, and there is no reason to expect they would behave differently on other classes of regulated parts — extending coverage is largely a matter of adding data.
- Guaranteed error rates need more data and compute. To guarantee the false-positive (below 0.01%, or 1 in 10,000) and false-negative (below 0.1%, or 1 in 1,000) rates one would want for deployment, we would need to train and evaluate on 10–100× larger datasets. We already have the necessary data; it is purely a matter of computational resources. Everything here was run on a personal laptop for safety — to avoid copying the real MCD data onto a shared lab server — but we have no doubt the target rates are achievable with even modest university-scale resources.
- No data governance yet. There is no shared, governed dataset: for others to train and validate their own detectors, we need a safe way to share this protected data, add new designs to it, and keep proper records.
- Novel designs are the hard case. The detectors recognize known designs and their disguises; a brand-new design absent from the catalog is the most likely real-world evasion, and would need methods that reason about whether a geometry could function as a device rather than whether it resembles a known one.