My intended open source project

I intend to create a digital image processor which is capable of locating specific structures from aerial photographs. Exact location and accuracy are NOT short term goals of this project, instead, the identification of obviously impervious areas will be stressed. Forecasted problems with this project include:

Discerning structures covered by trees
Many structures will have some tree cover. Hopefully this can be solved by requiring polygons and arcs of a reasonable size to be connected via different traces along what is supposedly the same structure.
Larger and industrial structure singularization
Large structures tend to have large irregularities located on the structure. This could be a problem when dealing with HVAC units, trailer parks, and street traffic. (I am including parking lots and streets in the definition of structures for those who find the last statement confusing.)
Ninjas could be a problem. Also I'll finish this list later