Automate your repostings using a simple python web scraper (90 min)

In this tutorial, we’re going to create a simple web scraper. Our bot will be able to scrape valuable content on any website, even if there’s no public API provided. Later, the gathered articles can be filtered and automatically (re)posted on social media or your blog.

My example bot is searching for new shopping deals related to a specific keyword on MyDealz.de and later posts the deal links on a public Facebook page. I encourage you to be creative! Don’t just blindly follow my tutorial, but try to adapt it to your use cases, other websites and social media platforms!

Simplified architecture of the bot

Overview: The bot lives in the looped main() function (mybot.py) that sequentially triggers the following methods. Every method will be discussed in a separate section of this tutorial.

1. fetch_articles() | scrape.py
2. filter_articles() | scrapy.py
3. persist_new_articles() | persist.py
4. post_new_articles() | post.py