Building a Manifest V3 Lead Extractor: Architecture of the Lead Extraction Engine

The Lead Extraction Engine is an open-source Chrome extension developed in Manifest V3 version 1.0.0 that extracts business-to-business and business-to-consumer leads from around 20 directory sites, including Google Maps, LinkedIn, Justdial, Yelp, Zillow, and IndiaMART. The extension's core functionality, which pulls leads into IndexedDB and exports them in CSV/JSON format, is encapsulated in the architecture described below.

The extension's manifest file specifies minimal viable permissions, including storage, scripting, activeTab, and notifications, as well as host permissions for all URLs. It defines a background service worker, which is crucial for the extraction process. The choice of "all_urls" for host permissions is a deliberate and somewhat aggressive one, as per the extension's README file. This allows the generic extractor to read contact information from arbitrary directory pages, where its primary function lies.

To ensure security, the extension omits unlimitedStorage and eliminates the use of remote code, eval, and external scripts, all of which are essential requirements for approval in the Chrome Web Store. The extraction pattern is implemented through a content/injector.js file that runs on every page. This file detects the platform and dispatches to one of approximately 20 extractor classes located in the content/extractors/ directory.

Each extractor class follows a consistent interface consisting of detect, parse, dedupe, and message functions.

For example, the LinkedIn extractor demonstrates this pattern well. It initializes with a set to store extracted hashes and a flag for auto-scrolling. The extractLeads method queries the document for specific search cards using a complex selector and extracts lead information from them. Parsing is site-specific, leveraging different CSS selectors for various directory sites.

Deduplication is achieved using a content hash derived from the name, title, and URL of the lead, hashed using btoa to a 32-character string. This ensures that the same lead is not counted twice, even if the user scrolls back to it.

The generic extractor, on the other hand, relies on a set of directory-shaped patterns to identify leads without needing site-specific knowledge. It uses recall-first design, meaning it captures leads before exporting them, and highlights the in-page extracted cards for user reference. The pipeline involves a background service worker that handles the SAVE_LEADS messages and validates the records before persisting them to IndexedDB.

This process occurs without any cloud involvement or user accounts, emphasizing the extension's offline capability.

The extraction process also includes a popup interface (popup.js, an ES module) that guides users through the auto-detection, starting and stopping of the extraction, and exporting the leads to CSV or JSON format. The monetization strategy is noteworthy, with a free version offering 50 leads offline, while paid tiers provide increasing numbers of leads.

Validation of these paid plans occurs server-side through a backend implemented in Express, using a POST endpoint at /validate secured by an X-API-Key. The extension never grants paid access from built-in keys; activation fails closed if the license server is not configured, which means that if the license server is not set up, activation simply does not work for paid plans, leaving the free tier untouched.

The server refuses to start without the LEE_API_KEY, which is provided in an .env.example file containing dummy values for testing purposes. The package can be tested by running npm test, which checks JavaScript syntax, manifest validation, and a hardcoded-secret scan.

To use the extension, users can load the source code directly from the Chrome extensions page (accessible at chrome://extensions) in Developer mode without requiring a build step or further compilation. The extension's source code can also be zipped for submission to the Chrome Web Store. The extension comes with a privacy policy and detailed explanations of permission requests, which are essential for compliance and transparency.

As of now, the Lead Extraction Engine is at version 1.0.0 and is not yet available on the Chrome Web Store.