A group of researchers is warning that popular AI image models like DALL-E 2 can be “tricked” into regenerating their training images.
That’s a serious privacy concern, especially as AI is applied to ever-more sensitive classes of images, such as in medical applications.
The team, which includes researchers from Google Brain, DeepMind, ETH Zurich, Princeton University, and University of California Berkeley, demonstrated that the class of image generators known as generative diffusion models memorise and regenerate their training data, something which “would violate all privacy guarantees” as well as raising questions about model generation and “digital forgery” (the model reproducing copyrighted works).
They tested the Stable Diffusion and Imagen models, and extracted “over a hundred near-identical replicas of training images that range from personally identifiable photos to trademarked logos”.
The paper, published on arXiv, “highlights the tension between increasingly powerful generative models and data privacy, and raises questions on how diffusion models work and how they should be responsibly deployed”.
For example, the researchers noted that fields like medical research are highly privacy-sensitive: the class of machine learning called a generative adversarial network (GAN) has already been applied to medical imagery, the paper said, which “underlines the importance of understanding the risks of generative models before we apply them to private domains.”
“Researchers and practitioners should be wary of training on uncurated public data without first taking steps to understand the underlying ethics and privacy implications," the paper said.
