Intelligence System
Project: Intelligence System
About
Challenge
I needed to design a scalable system capable of: Handling multiple sources (websites, Twitter, Facebook, Instagram) Bypassing dynamic/lazy-loaded content Extracting entities (people, places, dates) from raw data Archiving images, videos, and documents Preventing duplicate downloads and supporting daily automation
Goal
My goal was to: Automate scraping from public digital sources without using APIs Extract structured metadata from websites and social posts Build an archive system for downloaded content (media + text) Create a Flask-based dashboard with timeline search, entity filtering, and post review tools
Result
I used Playwright, BeautifulSoup, and Selenium to scrape and handle dynamic content. I integrated spaCy and regex for named entity recognition (people, dates, places). I stored structured outputs in JSON files and archived media in organized folders per website. I built a Flask web app to display entities in a searchable timeline interface.