The short answer is no, they are not worried, but they are concerned. Pilots have always had to deal with flying hazards. Pilots are accredited to looking outside the window and scanning for traffic and other conflicts. Up high, an aircraft has far fewer collision hazards to worry about. As the aircraft descends, there are more and more small airplanes. Go lower, and birds start to become a threat. Pilots start to sit up, keep their head on swivel and watch for dangers, wherever (or whatever) they might be.
With the recent report of a pilot spotting a small drone on approach to JFK many pilots are taking notice. Pilots are starting to do their research and find out what these drones are all about. Like the general public, many pilots are unaware of all the drone activity going on in the United States. They are finding out that lots of police departments and other local, state and federal agencies are flying these small surveillance drones. It seems that currently, most pilots feel that these small craft are being properly handled and managed. Probably because there are so few pilot reports of a near miss with a drone. Should that change sometimes in the future and reports start becoming more frequent, that feeling will change.
Prudent flight departments and airlines are monitoring the situation to ensure the FAA stay on top of the situation. While many pilots feel comfortable about the situation right now, they are concerned about the future. Anytime you increase air traffic, you increase the risk of mid-air collisions. The FAA is projecting a very large amount of drone traffic in the future over the US If these projections come true, collisions will become a very real threat, and they will occur. A small craft like the one that was spotted near JFK may cause little or no damage, or it could go into an engine inlet and cause complete power loss. A small craft (like this one) offer about the same danger as a large bird. Larger military drones, or (some of the proposed) cargo drones could cause catastrophic damage.
Flyers should know that today, there is very little danger from drones. Drones are a new phenomenon and as this industry evolves there are sure to be growing pains. The FAA is opening small test areas for further study on possible dangers. The results of these tests will aid in the development of strong regulations to control this additional traffic. One thing that appears certain is that drones are coming.