Here is the difference: Birds have an inate ability to see and avoid. A drone does not. Yes, many birds are hit every year, but there are a gazillion more birds than drones. That number is narrowing every day..
In my flying career I have run into several birds. Every one of them desperately trying to get away. Never any significant damage. I think you will agree that the largest bird you are likely to run into will weigh less than twenty pounds, but is made of soft, gooey stuff. Even then, the largest birds do significant damage. The smallest "toys" out there will do damage if hit. Genie is out of the bottle though, they are out there.