Viewing a single comment thread. View all comments

4

surreal wrote

why do you use selenium and not just http requests btw? selenium is pretty heavy

4

selver wrote

Selenium + Scrapy is just what I've used for this kind of stuff before (for stuff that did need selenium), so I can do it pretty fast. Didn't look into other options before coding it.

If you've got an easier way with http requests I'd love to see an example.