Go back to previous page
Forum URL: http://www.dombom.com/cgi-bin/dcforum/dcboard.cgi
Forum Name: The New MadBomber Marketing and SEO Forum
Topic ID: 44
Message ID: 0
#0, Web Crawler that Lists URL and and Pictures....
Posted by sgtaw on Dec-15-06 at 11:24 AM
Hi all,

Need some help.

I am an affilate of a merchant that sells hard goods. They don't have a datafeed so I am trying to create one myself.

I have found several programs that will crawl a site and list every single URL, keyword, description, and title. Then spit it out into csv or whatever.

Here's the problem. I also want it to grab the image urls on each page and associate it with that particular page. I know this will have multiple image urls picked up for each page, but I can clean that up easily.

If I can do this last step then having a full blown datafeed (with pictures is created!

Any suggestions?

Thanks,

Ed