Viewing a single comment thread. View all comments

4

selver wrote

I believe the issue was that those rss feeds keep reposting articles for some reason. Maybe every time they are edited? I didn't have much free time & stopped paying attention to it for a while, so I'm not sure.

I can try to fix the code up and throw together a subreddit crossposter later tonight.

4

surreal wrote

why do you use selenium and not just http requests btw? selenium is pretty heavy

4

selver wrote

Selenium + Scrapy is just what I've used for this kind of stuff before (for stuff that did need selenium), so I can do it pretty fast. Didn't look into other options before coding it.

If you've got an easier way with http requests I'd love to see an example.