Skip to content

z26zheng/web-crawler

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

19 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

web-crawler

TODOs

  1. The redfin crawler can't advance the search result pages. It's currently only scraping listings on Page 1.
  2. Scraping can get stuck and we need to solve two corner cases:
    1. After scraping 40 properties, a captch page will show up. We need to solve work with captcha.
    2. The page will pop up a login, we should either login, or refresh the page, or maybe this doens't matter to playwrite.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published