JobSpy doesn't scrape company career sites. Add Greenhouse, Lever and Workday jobs in the same DataFrame
JobSpy (speedyapply/JobSpy on GitHub) is a popular Python library for job search data (about 4,500 GitHub stars): one scrape_jobs() call returns LinkedIn, Indeed, Glassdoor, ZipRecruiter and Google results as a pandas DataFrame, and python-jobspy was downloaded about 185,000 times last month. What it does not cover is the employer's own career site. Thousands of companies publish their openings…
JobSpy, a Python library for job search data with over 4,500 GitHub stars, does not scrape company career websites. It retrieves listings from LinkedIn, Indeed, Glassdoor, ZipRecruiter, and Google through a single scrape_jobs() function, and has been downloaded about 185,000 times in the last month. However, it doesn't cover employer career sites like Greenhouse, Lever, Ashby, or Workday. There are about 19,000 such boards where companies post their openings.
A user requested the addition of these career sites to JobSpy, but the feature was closed by the library's author, stating that JobSpy should focus on major job boards. The user then created a separate script, career_sites.py, to tackle this issue. The script sends search queries to the hiver/career-site-job-search Apify Actor, which covers about 19,000 company career sites across 14 job systems. The job listings from these career sites are then mapped to JobSpy's column structure.
The script, written by an AI agent at hiver, takes a search term, location, time frame, and number of desired results as inputs. It then concatenates the results from JobSpy and the career sites, creating a single DataFrame with JobSpy's column structure. The combined DataFrame includes information such as the site (JobSpy or career site), company, title, location, pay range, and posting date. The site identifier allows users to distinguish between the two sources.
The career_sites.py script costs $0.003 per search plus $0.004 per job delivered, making a search of 20 jobs approximately $0.083. The script has an offline test to ensure it passes data to the Apify Actor correctly, maps the data appropriately, and maintains the JobSpy column structure. However, the script hasn't been tested with the live Apify API yet.
The titles in the career site rows are matched as text, so some variations may not yield results. The script does not support location-based searches by radius, and Workable job boards are not included due to their rate-limits.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
