Viewing a single comment thread. View all comments


Tequila_Wolf wrote

Thanks! Presently we have some code, here.

We've had to stop it from posting because it reposts a lot though - we had u/ThreadBot post to f/itsgoingdown and u/CrimeBot f/CrimethInc but it was posting each post so many times it wasn't viable - I'm not sure why. It might be that the RSS was republishing the articles? If you want to look at / work on that, that'd be cool! Might want to check in with u/selver, who I think put together the original code here?

Unfortunately I know little about how this stuff works.


selver wrote

I believe the issue was that those rss feeds keep reposting articles for some reason. Maybe every time they are edited? I didn't have much free time & stopped paying attention to it for a while, so I'm not sure.

I can try to fix the code up and throw together a subreddit crossposter later tonight.


surreal wrote

why do you use selenium and not just http requests btw? selenium is pretty heavy


selver wrote

Selenium + Scrapy is just what I've used for this kind of stuff before (for stuff that did need selenium), so I can do it pretty fast. Didn't look into other options before coding it.

If you've got an easier way with http requests I'd love to see an example.