.NET Development

Software
Services
July 14, 2011

Multithreading crawler with synchronized processes- ISpider- a crawler process scan the websites and gathered  all real estate ads in order to accomplish a database.
A second thread of the crawler load each ads page, copy the html source from the website in order to extract the necesarry information to accomplish the requirements for the real estate database using regular expresions.
Rescue in the database is managed by a WCF service, hosted in a windows service.
There is also a manager proces which manages scheduling, geographic area for which the crawler is activated and enables activation / deactivation of individual / collective of the crawling processes.

Leave a Reply

Your email address will not be published. Required fields are marked *