in ,

Curryfinger – Find the Server Behind the CDN, Hacker News

Unix philosophy your way to finding the real host behind the CDN.

Travis dropped

CURRYFINGERmeasures a vanilla request for a particular URL against requests directed to specific IP addresses with forced TLS SNI and HTTP Host headers. The tool takes a string edit distance, and emits matches according to a rough similarity metric threshold.

There are many guides that explain the process of finding servers that may actually host a CDN fronted domain, which all boil down to;

  • Plug the domain name into$ OSINTTool; shodan, censys, etc.
  • Collect IP addresses.
  • ????
  • Profit

“But Travis,” you say “we already have a tool for this, why do we need yet another one?”

Many guides point to an open source tool,christophetd / CloudFlair, that roughly does this;

  • Downloads CloudFlare’s IP ranges.
  • Checks whether a supplied domain resolves to an IP within those ranges.
  • Queries the Censys API for IPs serving X. 509 certificates with the provided domain in theCN=(CommonName) attribute.
  • Loads each IP, and compares the result against a control.

Unfortunately,cloudflair.pyis a little slow, and it fails to indentify true-positives in many cases. Concretely, downloading CloudFlare’s IP lists on every run compounds already slowpythonwarm-up times – and, maybe more importantly ,cloudflair.pywill not work on non-CloudFlair CDNs.

Why not just commit to an existing project? One; Python has its uses, but writing highly performant multi-threaded scanners is not one of them. Two; we get value from separating the concerns of identifying targets and verifying them, to try other, more egregious methods at finding candidate origin servers than commercial OSINT platforms.

CURRYFINGERdemonstrates the kind of effective PoC you can pump out in a few hours using Golang. It has been battle tested against thousands of domains, across hundreds of thousands of requests, and run on dozens of servers. I’ll share that information in another post, but let’s just take a look at one example;

Here, we putcloudflair.pyup againstCURRYFINGERin an attempt to identify the real server behind the popular “chat”

Left Pane – CloudFlair

We launch./ -o chatbate.txt– kicking off the process of finding targets and carrying out similarity analysis.


We find targets by querying the Shodan REST API;curl "https: // key=$ SHO & query=ssl% 3A " "" | jq ".matches |. []. ip_str" | tr -d "" t "| tee

Then we invokeCURRYFINGERon the results to find which IPs seem like the real origin servers behind the CDN;./ CURRYFINGER -file -show=false -url 2>/ dev / null | tee res.txt

Then we drop the CloudFlare IP addresses from the results;grep ^ match res.txt | grep -v 104. 16 | cut -d "" -f 2

We finally manually examine the full response by forcingcurlto resolve a domain with a specific IP;curl -vik --resolve 443: $ IP https: // chaturbate .com


So What

tl; dr;cloudflair.pyis still running afterCURRYFINGERcompletes and we’ve verified the results. By the timecloudflair.pyfinishes, it has failed to identify the correct server, even thoughCensys found the IP, andcloudflair.pychecked it.

./ CURRYFINGER -h ... - FINE DUAL USE TECHNOLOGIES Usage of ./CURRYFINGER:   -file string     readips from specified -file instead of stdin.   -mbits int     Match in the first -mbits.(default500)  -perc int     Match at -perc[entage]Similarity( (default)  50)  -show     Show sample responses.   -threads int     Number of -threads to use.(default200)  -timeout duration     Timeout the check.(default  (s) )  -ua string     Specify User Agent, otherwise we'll generate one.   -url string     -url to check.(default"") )

The CURRYFINGER help text .

- file string
You can specify IP addresses to test viastdin, or you can throw a filename here.
– mbits int
This is the number ofbyteswe’ll consider out of the replies from servers. 500 Bytes is a good default. You can bump this up if you get too many false positives.
– perc int
We divide the total examined bytes by levenshtein edit distance, and call that a ‘percentage’ fun fact; the edit distance can exceed the original sample. It works well enough as a measurement, and empirical results over 15, 000 hits show roughly show the 25 th percentile at- perc 74. Our default of 50 is good.
– show bool
Setting- show=truewill emit both measurement samples tostderr, which is fine for debugging, but you’ll want to set this to- show=false.
– threads int
How many simultaneous threads will be used to perform requests. I’ve used up to fifty-thousand concurent threads over thousands of ips. It works just fine.
– timeout duration
This timeout applies to the total connection to a target server. The default timeout is extremely conservative, values ​​down to- timeout 1sare just fine. If you’re saturating your pipe with- threads 500000then you ‘re going to want to increase timeout, or decrease threads. YMMV.
– ua string
We usually generate a random User Agent string for requests, but you can specify one here. I wouldn’t.
– url string
Thehttps: //prefixed url we’re going to grab for our tests.

****************************************** (Getting IP Addresses)

If you have a free Shodan account, you have an API Key;

export(SHO)=[YOUR SHODAN API KEY] curl" search? key=$ SHO& query=ssl% 3A  "$ DOMAIN ""|JQ". matches |. []. ip_str"|tr -d""  t "|tee targetIPs .com.txt

Grab some IPs

You can also grab CIDR ranges for popular cloud hosting providers, andmasscan -p 443them. I’ll explore this option in another article.


CURRYFINGERdoes full connects, and doesn’t know what yourulimits are. So, juice those up before a run;ulimit -n 60000. Yep.

VHOST check; lots of domains, just a few IPs

With a pile of IP addresses a pile of domains intargetDOMAINS.txtyou can quickly test for the presence of every domain on every IP by using GNUparallel.

(parallel -j)  20./CURRYFINGER -url https: //{}-threads200-show=(false)  -timeout 3s -file :::: targetDOMAINS.txt  (2)>/ dev / null|grep ^ match|tee results.txt
Vanilla application of GNU Parallel

All together now; match subdomains

Pull subdomains for a target domain before runningCURRYFINGERnow you’re cooking with concentrated freedom. Of course, use whatever tools you want,amass,subbrute, Censys, Shodan,masscan, whatever.


Set up env vars .

Here’s what that looks like;

python -e ssl, ask, bing, google, yahoo, netcraft, dnsdumpster, virustotal, threatcrowd, passivedns -d$ DOMAIN-o Fix newlines|  (tr -d)  " r">>targetDOMAINS.txtecho$ DOMAIN>>targetDOMAINS.txt

Grab subdomains

Curl  "$ SHO& query=ssl% 3A  "$ DOMAIN ""|JQ". matches |. []. ip_str"|  (tr-d)  ""  t "|tee -j400./CURRYFINGER -url https: / /{}-threads200-show=(false)  -timeout 3s -mbits5000-file :::: targetDOMAINS.txt2>/ dev / null|grep ^ match|tee results.txt

Let it rip with 400 parallel instances of CURRYFINGER and match against more bytes.


Brave Browser
Read More

What do you think?

Leave a Reply

Your email address will not be published. Required fields are marked *

GIPHY App Key not set. Please check settings

Bahamas brace for Tropical Storm Humberto just weeks after Hurricane Dorian – ITV News,

Bahamas brace for Tropical Storm Humberto just weeks after Hurricane Dorian – ITV News,

The hidden dangers of magnet fishing, Hacker News

The hidden dangers of magnet fishing, Hacker News