Bulk tech stack lookup: check thousands of domains at once

Your options at a glance

Prices for the paid services were checked on 2026-10-03 and change often; treat them as a rough guide.

The extension sees more than any batch tool on a single site, because it runs JavaScript in your browser and watches inner pages. Batch tools trade that depth for speed: a raw homepage fetch plus DNS takes well under a second per domain, so a list of thousands finishes in minutes.

Do it yourself with free command-line tools

If you are happy in a terminal, ProjectDiscovery's open-source httpx detects technologies with the open Wappalyzer-format fingerprints. Put one domain per line in domains.txt:

httpx -l domains.txt -tech-detect -follow-redirects -json -silent > tech.jsonl

Each line of tech.jsonl is a JSON object with a tech array. Our test on 2026-10-12 (httpx 1.12.0) returned this for wordpress.org:

"tech": ["Google Tag Manager", "Gutenberg", "HSTS", "MySQL", "Nginx", "PHP",
         "WordPress Block Editor", "WordPress Site Editor", "WordPress:7.2"]

Without -follow-redirects, sites that redirect (shopify.com answers 301) only show the redirect's technologies. For the email and certificate side, standard tools work one domain at a time:

dig +short MX example.com                 # email provider
dig +short TXT example.com | grep spf1    # SPF and its include: services
dig +short TXT _dmarc.example.com         # DMARC policy: p=none, quarantine or reject
echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null \
  | openssl x509 -noout -issuer -enddate  # certificate issuer and expiry

This gets you far for a few hundred domains. What you end up building yourself: merging four tools into one row per domain, following SPF include: chains to count DNS lookups (the limit is 10), grading DMARC and security headers, retrying flaky sites, reading robots.txt and llms.txt, and keeping fingerprints up to date.

Run a bulk lookup without code

1. Open Bulk Tech Stack Lookup on Apify and sign in (a free account works).
2. Paste your list into Domains or URLs. Domains, full URLs and even email addresses are accepted; only the domain part is used and duplicates are removed.
3. Leave the checks on, or switch off the ones you do not need (faster, not cheaper).
4. Click Start. About 20 domains are checked in parallel by default.
5. Open the Output tab and export CSV or Excel, or pick one of the four views.

The same run as JSON input:

{
  "domains": [
    "apify.com",
    "https://www.shopify.com",
    "ops@stripe.com",
    "wordpress.org"
  ],
  "maxConcurrency": 20,
  "minConfidence": 50
}

One row from a real run on 2026-10-03, shortened:

{
  "domain": "wordpress.org",
  "status": "ok",
  "cms": "WordPress",
  "technologyNames": ["WordPress", "WordPress Block Editor", "Gutenberg", "MySQL", "PHP", "Nginx",
                      "Google Font API", "Google Tag Manager", "HSTS", "Let's Encrypt", "Open Graph", "RSS"],
  "emailSecurityGrade": "A",
  "securityHeadersGrade": "E",
  "tlsDaysUntilExpiry": 79,
  "hasLlmsTxt": true,
  "dns": {
    "mx": ["smtp1-dca.wordpress.org", "smtp2-dca.wordpress.org"],
    "spf": {"policy": "fail", "includes": ["helpscoutemail.com"], "dnsLookupCount": 1},
    "dmarc": {"policy": "reject", "pct": 100, "reportingVendors": ["dmarcian"]},
    "sendingServices": ["Help Scout"]
  },
  "tls": {"issuer": "Let's Encrypt", "notAfter": "2026-12-22T19:43:57+00:00", "lifetimeDays": 90}
}

Worked example: what 56 well-known sites run

We ran a list of 59 inputs (big SaaS, ecommerce, developer and news sites, plus a few Bulgarian sites and deliberately broken entries) on 2026-10-03. 56 domains answered, 1 did not exist and 2 inputs were invalid. Among the 56:

Two lessons for reading your own results:

• "Expires soon" is a weak signal on its own. Three healthy sites had 20 to 28 days left, on certificates with lifetimes of 90, 199 and 317 days. Lifetimes keep shrinking under CA/Browser Forum rules (200 days maximum since March 2026), and big sites renew automatically, often late. Stronger signals are tls.valid = false or a negative tlsDaysUntilExpiry; the one expired certificate in this list was badssl.com's deliberate test certificate.
• A blank CMS is not "no CMS". Headless and custom front ends often leave no trace in the raw HTML. Combine cms with technologyNames and hosting before you segment.

That run used the actor's earlier browser-like User-Agent. Since 2026-10-04 it identifies itself honestly and honours robots.txt, so a few of these sites would now come back as blocked_by_site or disallowed_by_robots_txt (free rows, with DNS, email and TLS data but no tech stack).

Turn the rows into prospect lists

Export the Overview view (flat columns) or the Email & DNS security view to a spreadsheet and filter:

Schedule the same list weekly and compare runs to catch changes: a new CMS version, a DMARC policy moving to reject, or a certificate that stopped renewing.

What each row contains

Honest limits

• Homepage only, no JavaScript. Tools that load after client-side rendering or only on inner pages (a checkout script, a chat widget on /contact) can be missed. Wappalyzer's js and xhr rules are skipped for the same reason.
• DNS verification is not usage. detectedBy: ["dns:TXT"] means the company verified the domain with that SaaS (Atlassian, OpenAI, DocuSign…). It does not prove the tool runs on the site.
• Bot-protected sites are not bypassed. The actor sends an honest User-Agent, follows robots.txt and uses no proxy. Sites that answer 403, keep answering 429 or serve a challenge page come back as blocked_by_site, with DNS, email and TLS data but no tech stack.
• Fingerprints are community-maintained. Detection uses the open webappanalyzer set (7,000+ technologies, pinned to a version for reproducible results). Rare or brand-new tools may be missing.
• No personal data. It reads headers, HTML, public DNS and certificates. It does not collect names, emails or phone numbers; security.txt is reported as present or absent only.

All input fields

Generated from the actor's live input schema (build 1.0.3).

Call it from code

Start a run and get the results in one HTTP request with the Apify API (your token is under Console, Settings, API & Integrations):

curl -X POST "https://api.apify.com/v2/acts/adamhaleworks~bulk-tech-stack-lookup/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"domains":["apify.com","wordpress.org"]}'

The run's Output tab offers 4 ready-made views; every view exports to CSV, Excel, JSON or XML:

• Overview: domain, status, finalUrl, title, cms, cdn, hosting, emailProvider, dnsProvider, emailSecurityGrade, securityHeadersGrade, tlsDaysUntilExpiry, blocksAiTraining, hasLlmsTxt, technologyNames, error
• Email & DNS security: domain, emailProvider, dns.sendingServices, emailSecurityGrade, dns.spf.policy, dns.spf.dnsLookupCount, dns.spf.exceedsLookupLimit, dns.dmarc.policy, dns.dmarc.reportingVendors, dns.hasMtaSts, dns.hasBimi, dns.caaIssuers, dns.saasVerifications
• TLS & headers: domain, tls.valid, tls.issuer, tls.notAfter, tls.daysUntilExpiry, tls.lifetimeDays, tls.hostnameMatches, tls.protocol, securityHeaders.score, securityHeaders.grade, securityHeaders.missing, securityHeaders.versionLeaks, wellKnown.securityTxt.found
• AI crawler policy: domain, aiCrawlers.robotsTxtFound, aiCrawlers.blocksAllAiTraining, aiCrawlers.blockedBots, aiCrawlers.namedBots, aiCrawlers.contentSignals, aiCrawlers.llmsTxt.found, aiCrawlers.llmsFullTxt.found

What it costs

Once pricing is switched on, only rows with status: ok are charged. Dead domains, unreachable sites, blocked or robots.txt-disallowed sites and invalid inputs are written to the dataset for free, so cleaning a lead list costs nothing for the dead entries.

FAQ