Oct 03

Adventures in geekdom – harvesting referrals using PERL

Proven By SciencePost a comment • 425 views

It’s all very well getting the recent 100 referrals on SIteMeter but unless you stump up the cash, that’s as much as sitemeter will let you see, and IT’S NOT ENOUGH. I crave referral stats, I want to keep the data FOREVER. So as the FT server can’t be set up to do it either, I contrived a way to let my mac do the equivalent of continually refreshing the above link. 

I have a perl script cron’d to run every 3 or 4 minutes that includes this code:

$_ = `curl -s "http://www.sitemeter.com/default.asp?blahblah"`; while (/default.asp?action=stats&site=ftrig&visit=([^&]*)&report=9/g) {      $session_id=$1;        /size="2">(.*?)

which parses the HTML of the referral page to find all 20 of the listed referral URLs and session_ids (an integer from 1 to 100). When a session_id pops up that wasn’t in the list last time, it is USUALLY a new session, so log the referral. (Perl fans: you could do a list-context “/sg” match to get all the results at once, but you’d still need to loop through the list anyway.) 

There, I hope you enjoyed that – I’m still not entirely sure it’s appropriate. And if this gets our sitemeter accounts shutdown I am sincerely sorry.

Add your comment

(Register to guarantee your comments don't get marked as spam.)


Required (Your email address will not be published)

Top of page