- 定义管道类(在pipelines.py里定义)
import urllib.request
class DangDangDownloadPipelines:
def process_item(self, item, spider):
url = 'http:' + item.get('src')
filename = '../books_img/' + item.get('name') + '.jpg'
urllib.request.urlretrieve(url, filename)
return item
- 在settings中开启管道
模仿之前的管道写一个放到ITEM_PIPELINES里就好了
'scrapy_dangdang_39.pipelines.DangDangDownloadPipelines': 301