Last Friday, October 4th 2013 I was hanging out with some friends and while searching for a place to do so, I opened up Urbanesia on it mobile web http://m.urbanesia.com/. While I was searching, I noticed there’s a banner displayed on top of the page, it was definitely not Urbanesia’s and so I got curious.
When digging into the mobile web source code, we didn’t activated any codes that’ll lead to any banner placements on top so this banner must surely be injected by a third party. My mind were full of technical ideas translating into security breaches/flaws making the injection a reality.
First of all, we didn’t use any JavaScript library on our mobile web, it was all pure JavaScript. All the JavaScript and CSS the website needed were embedded into the HTML. The website is very light and yet it took a relatively extended period of time to load on my Nexus 4. So I switched to opening it with my Z10 and it loaded as fast as I expected it to be. There wasn’t any banner, I was using a Telkomsel 3G connection on my Z10.
So the culprit is XL definitely since they also took the time to display their logo on the banner. Here’s a screenshot.
This is most definitely a low blow by XL. They’re practically monetizing using our content and also displaying ads on a 3G connection I already paid for. For both the content provider and user, this is a very unethical and disgraceful act to say the least.
My blog post here will talk more about the technicalities on how XL is able to do this. I won’t be 100% correct but I’ll settle for 99%.
XL’s technique has a flaw. Try refreshing Urbanesia’s mobile web and soon you’ll notice that in some cases, it will show code fragments instead of the website. Why? Urbanesia’s mobile uses Google Analytics server side implementation to track our statistics. By doing so, we have a small ga.php on our server to track users. Here are the codes below.
| <?php | |
| /** | |
| Copyright 2009 Google Inc. All Rights Reserved. | |
| **/ | |
| error_reporting(0); | |
| function __detectVisit() { | |
| $agent = strtolower($_SERVER['HTTP_USER_AGENT']); | |
| $bot_strings = array( | |
| "google", "bot", "yahoo", "spider", "archiver", "curl", | |
| "python", "nambu", "twitt", "perl", "sphere", "pear", | |
| "java", "wordpress", "radian", "crawl", "yandex", "eventbox", | |
| "monitor", "mechanize", "facebookexternal", "bingbot", | |
| "googlebot-image", "googlebot" | |
| ); | |
| foreach($bot_strings as $bot) { | |
| if(strpos($agent, $bot) !== false) { | |
| return "bot"; | |
| } | |
| } | |
| return "normal"; | |
| } | |
| // Tracker version. | |
| define("VERSION", "4.4sh"); | |
| define("COOKIE_NAME", "__utmmobile"); | |
| // The path the cookie will be available to, edit this to use a different | |
| // cookie path. | |
| define("COOKIE_PATH", "/"); | |
| // Two years in seconds. | |
| define("COOKIE_USER_PERSISTENCE", 63072000); | |
| // 1x1 transparent GIF | |
| $GIF_DATA = array( | |
| chr(0x47), chr(0x49), chr(0x46), chr(0x38), chr(0x39), chr(0x61), | |
| chr(0x01), chr(0x00), chr(0x01), chr(0x00), chr(0x80), chr(0xff), | |
| chr(0x00), chr(0xff), chr(0xff), chr(0xff), chr(0x00), chr(0x00), | |
| chr(0x00), chr(0x2c), chr(0x00), chr(0x00), chr(0x00), chr(0x00), | |
| chr(0x01), chr(0x00), chr(0x01), chr(0x00), chr(0x00), chr(0x02), | |
| chr(0x02), chr(0x44), chr(0x01), chr(0x00), chr(0x3b) | |
| ); | |
| // The last octect of the IP address is removed to anonymize the user. | |
| function getIP($remoteAddress) { | |
| if (empty($remoteAddress)) { | |
| return ""; | |
| } | |
| // Capture the first three octects of the IP address and replace the forth | |
| // with 0, e.g. 124.455.3.123 becomes 124.455.3.0 | |
| $regex = "/^([^.]+\.[^.]+\.[^.]+\.).*/"; | |
| if (preg_match($regex, $remoteAddress, $matches)) { | |
| return $matches[1] . "0"; | |
| } else { | |
| return ""; | |
| } | |
| } | |
| // Generate a visitor id for this hit. | |
| // If there is a visitor id in the cookie, use that, otherwise | |
| // use the guid if we have one, otherwise use a random number. | |
| function getVisitorId($guid, $account, $userAgent, $cookie) { | |
| // If there is a value in the cookie, don't change it. | |
| if (!empty($cookie)) { | |
| return $cookie; | |
| } | |
| $message = ""; | |
| if (!empty($guid)) { | |
| // Create the visitor id using the guid. | |
| $message = $guid . $account; | |
| } else { | |
| // otherwise this is a new user, create a new random id. | |
| $message = $userAgent . uniqid(getRandomNumber(), true); | |
| } | |
| $md5String = md5($message); | |
| return "0x" . substr($md5String, 0, 16); | |
| } | |
| // Get a random number string. | |
| function getRandomNumber() { | |
| return rand(0, 0x7fffffff); | |
| } | |
| // Writes the bytes of a 1x1 transparent gif into the response. | |
| function writeGifData() { | |
| global $GIF_DATA; | |
| header("Content-Type: image/gif"); | |
| header("Cache-Control: " . | |
| "private, no-cache, no-cache=Set-Cookie, proxy-revalidate"); | |
| header("Pragma: no-cache"); | |
| header("Expires: Wed, 17 Sep 1975 21:32:10 GMT"); | |
| echo join($GIF_DATA); | |
| } | |
| // Make a tracking request to Google Analytics from this server. | |
| // Copies the headers from the original request to the new one. | |
| // If request containg utmdebug parameter, exceptions encountered | |
| // communicating with Google Analytics are thown. | |
| function sendRequestToGoogleAnalytics($utmUrl) { | |
| $options = array( | |
| "http" => array( | |
| "method" => "GET", | |
| "user_agent" => $_SERVER["HTTP_USER_AGENT"], | |
| "header" => ("Accepts-Language: " . $_SERVER["HTTP_ACCEPT_LANGUAGE"])) | |
| ); | |
| if (!empty($_GET["utmdebug"])) { | |
| $data = file_get_contents( | |
| $utmUrl, false, stream_context_create($options)); | |
| } else { | |
| $data = @file_get_contents( | |
| $utmUrl, false, stream_context_create($options)); | |
| } | |
| } | |
| // Track a page view, updates all the cookies and campaign tracker, | |
| // makes a server side request to Google Analytics and writes the transparent | |
| // gif byte data to the response. | |
| function trackPageView() { | |
| $timeStamp = time(); | |
| $domainName = $_SERVER["SERVER_NAME"]; | |
| if (empty($domainName)) { | |
| $domainName = ""; | |
| } | |
| // Get the referrer from the utmr parameter, this is the referrer to the | |
| // page that contains the tracking pixel, not the referrer for tracking | |
| // pixel. | |
| $documentReferer = $_GET["utmr"]; | |
| if (empty($documentReferer) && $documentReferer !== "0") { | |
| $documentReferer = "-"; | |
| } else { | |
| $documentReferer = urldecode($documentReferer); | |
| } | |
| $documentPath = $_GET["utmp"]; | |
| if (empty($documentPath)) { | |
| $documentPath = ""; | |
| } else { | |
| $documentPath = urldecode($documentPath); | |
| } | |
| $account = $_GET["utmac"]; | |
| $userAgent = $_SERVER["HTTP_USER_AGENT"]; | |
| if (empty($userAgent)) { | |
| $userAgent = ""; | |
| } | |
| // Try and get visitor cookie from the request. | |
| $cookie = $_COOKIE[COOKIE_NAME]; | |
| $guidHeader = $_SERVER["HTTP_X_DCMGUID"]; | |
| if (empty($guidHeader)) { | |
| $guidHeader = $_SERVER["HTTP_X_UP_SUBNO"]; | |
| } | |
| if (empty($guidHeader)) { | |
| $guidHeader = $_SERVER["HTTP_X_JPHONE_UID"]; | |
| } | |
| if (empty($guidHeader)) { | |
| $guidHeader = $_SERVER["HTTP_X_EM_UID"]; | |
| } | |
| $visitorId = getVisitorId($guidHeader, $account, $userAgent, $cookie); | |
| // Always try and add the cookie to the response. | |
| setrawcookie( | |
| COOKIE_NAME, | |
| $visitorId, | |
| $timeStamp + COOKIE_USER_PERSISTENCE, | |
| COOKIE_PATH); | |
| $utmGifLocation = "http://www.google-analytics.com/__utm.gif"; | |
| // Construct the gif hit url. | |
| $utmUrl = $utmGifLocation . "?" . | |
| "utmwv=" . VERSION . | |
| "&utmn=" . getRandomNumber() . | |
| "&utmhn=" . urlencode($domainName) . | |
| "&utmr=" . urlencode($documentReferer) . | |
| "&utmp=" . urlencode($documentPath) . | |
| "&utmac=" . $account . | |
| "&utmcc=__utma%3D999.999.999.999.999.1%3B" . | |
| "&utmvid=" . $visitorId . | |
| "&utmip=" . getIP($_SERVER["REMOTE_ADDR"]); | |
| sendRequestToGoogleAnalytics($utmUrl); | |
| // If the debug parameter is on, add a header to the response that contains | |
| // the url that was used to contact Google Analytics. | |
| if (!empty($_GET["utmdebug"])) { | |
| header("X-GA-MOBILE-URL:" . $utmUrl); | |
| } | |
| // Finally write the gif data to the response. | |
| writeGifData(); | |
| } | |
| ?><?php | |
| if(__detectVisit() === "normal") | |
| trackPageView(); | |
| ?> |
The script displays a small 1x1 GIF file and I made some modifications to only track users and not bots. Now instead of showing the GIF file, it showed code fragments. They mistakenly thought the ga.php file for a document when in fact it’s an image. The Content-Type was correct as image/gif but I think they filter by looking at its file extension. Here’s the screenshot.
There were some JavaScript URLs there so I went on to download the JavaScript file. The file is only downloadable if you’re on their 3G network. They have a minified and obfuscated version named ibn_complete_20130930.min.js and a more readable version named ibn_complete_20130930.js. Here’s a Github Gist of the readable version: https://gist.github.com/tistaharahap/6837508.
You’ll notice that they included their development and production URLs intact within the source code. They also included jQuery with the codes. When looking at the codes, they were only doing minimal DOM manipulation using jQuery and yet the bloated user’s download size with the whole source code.
So my attention went into the JavaScript file they served. To disable the ad serving mechanism, I thought I can just nullify any requests to their ad servers URLs. So I tried to do so here. It didn’t work which gets me even more curious. This leaves me only 1 option, to debug straight into my Nexus 4 by following the instructions from Google here.
What got me more pissed off than before, I found out that Urbanesia’s content was served as an IFRAME inside their HTML document. There were placeholders for top, left, right and bottom banners. But if you see the screenshots above, the domain of the website is still Urbanesia’s. So basically XL did not only modified contents, they also manipulate users into believing that the ads are coming from Urbanesia’s server. This disgusts me.
You see above, it’s ridiculously inappropriate at all. Urbanesia did not consent to anything like this at all nor there were any notifications to our part about their practices. This is just wrong.
My tweets were responded by friends in the startup community and also from outside the community. The responds were in Facebook, I’m auto posting tweets to Facebook by the way. I can’t represent them as this blog post is my personal thoughts but suffice to say, my thoughts are not much different than theirs.
So whether you’re a content provider or a user, we are both offended by this kind of practice by XL. The big boys said that they are writing a protest letter to XL, let’s see how this can stop XL from violating our rights. Yes we live in Indonesia which gets me even more pissed off. If you wanna help, retweet this blog post and let more people know.
DISCLAIMER: While I work at Urbanesia, this blog post is my personal thoughts and therefore does not imply the views of Urbanesia as an entity.