This repo is the shortened, generalized, and more reusable version of the Twitter Scraping tool I developed for research analysis of the environmental action situation in the UK. Contact for the original full version.
Simplified tool for scraping Twitter (also known as X) using Selenium webdriver.
Logs in by provided username and password, and then goes to a provided search URL, giving the date, statistics, and contents of any number of tweets associated with the search.
After downloading, install and add chromedriver: https://chromedriver.chromium.org/downloads
You will need Selenium and Python.
You will also need a Twitter account to run the bot on.
To use the program, run main.py and provide the details described in the tips below.
Running it will prompt for username, password, minimum, and query.
username & password: These must be the credentials of your twitter account. It is recommended that you use a throwaway account.
minimum: Minimum refers to the number of tweets you want to scrape. Enter an integer.
query: The search page link. You can go to Twitter, enter a search query (or use advanced search), and copy the resulting page URL.