MWC Milestone 0.2 has been Reached

MWC has reached a new milestone v0.2.

All milestones are important and this one is no different, MWC is now in a working state, though not a perfect one as there are so many edge cases in the world wide web, but MWC can be easily started, it’s internal code structure makes sense, and the most basic crawling features are implemented.

All of the above being said, there is still so much work to be done till MWC will be in a presentable state.

Basic features should still be polished, exception handling should be upgraded, some 3rd party implementations of the main components should be added and of course there is the matter of advanced features (Proxy support, authentication support, proper politeness support and many more).

Anyway, this is a start, and a good one at that, so I am pleased

Modular Web Crawler