6pm-scraper

Initial Setup

pip install scrapy
start project by typing following command
scrapy crawl 6pm-crawl -o products.json
The products will be stored in products.json file but you can store into database by using scrapy pipelines.
See scrapy documentation for more

Name		Name	Last commit message	Last commit date
Latest commit History 14 Commits
spiders		spiders
.gitignore		.gitignore
README.md		README.md
constants.py		constants.py
items.py		items.py
middlewares.py		middlewares.py
pipelines.py		pipelines.py
settings.py		settings.py