I recently came across a vendor selling a drum brake where they claimed it had "No asbes" i.e. no asbestos. This led me to search about asbestos in bicycle brakes, and indeed it's a thing:
http://santanatandem.com/Library/AraiDBrakeIns.pdf http://santanatandem.com/Library/AraiDBrakeIns.pdf (go to page 2, paragraph 4 in "Service")
The quantity is small and it's not friable, it's solid chunks (like old chemistry lab benches) which pose no actual hazard. Unless you are frequently overhauling these drum brakes with abrasives & sucking in the dust, the risk should be negligible, but it's good to know about.
If any shops out there have a bin for drum brakes, I'd suggest putting a warning label on it, or attaching a tag to the individual brakes if they are in a mixed bin.