In this video, I'll be showing you how to build an AI web scraper using Python. The application itself is super cool as it scrapes the site based on the URL you give it, grabs the DOM content, and allows us to then pass a prompt to the AI that will be able to grab information from the website that we want. In addition to this quick demo of how everything works, I will also take a deep dive into how to build the AI web scraper utilizing tools such Selenium, BeatifulSoup, LangChain, & more!

Get a free credit when you sign up for BrightData with a new account: https://brdta.com/techwithtim_ai

If you want to land a developer job check out my program with CourseCareers: https://techwithtim.net/dev

🎞 Video Resources 🎞
Code In This Video: https://github.com/techwithtim/AI-Web...
ChromeDriver Download: https://googlechromelabs.github.io/ch...
Selenum Video Tutorial:    • Python Selenium Tutorial - Automate W...  
Ollama Download Link: https://ollama.com/
Ollama Github: https://github.com/ollama/ollama

⏳ Timestamps ⏳
00:00 | Overview
00:15 | Install/Setup
04:30 | Starting the Streamlit UI
07:34 | Web Scraping with Selenium
15:59 | Unblocking Websites/Captchas/IP Bans
22:30 | Cleaning DOM Content
31:10 | Parsing Content with Ollama (LLM)
39:45 | Ollama Installation
42:45 | Testing the Scraper

Hashtags
#python #ai #webscraping