Serverless Web scrape using AWS Lambda and S3 — Python

  • Get data from a website, store it in a dataframe. In this case, I’m looking for certain words of interest from news headlines.
  • Save it down to csv
  • Run this every day (using Cron) to see interesting things in how the data changes over time
Photo by Ben White on Unsplash
functions:
scraper:
handler: scraper.handler
boto3
pandas
bs4
sls deploy

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
John Calabrese

John Calabrese

91 Followers

Focused on outcomes that build value. Passionate about finance, risk, data and blockchain.