Measuring download speed by embedding a GIF inside a JavaScript comment
As part of my day job I've been spidering a large number of web sites. While debugging the spider to make sure that its results were consistent I came across a curious web page that contains binary data inside a comment inside <script> tags.
The spider was simply requesting http://www.usaa.com/ which then goes through a sequence of complicated redirects. Here's the output from the excellent HTTP debugging tool HTTPFox:
The critical page to watch is https://www.usaa.com/inet/ent_logon/Logon?redirectjsp=true. When loaded with no cookies it is served containing JavaScript code that captures information about the browser and posts it back to the server:
The interesting part is the function getConnectionSpeed() which is written as follows:
And that relies on knowing a startTime and endTime. They is done by embedding a 10k GIF inside the web page, not by loading it, but by embedding it inside a JavaScript comment. The time is measured at the start and end of the GIF being loaded into the browser and the bandwidth determined from that.
Here's a shortened (and wrapped) version of the code:
Now the GIF data looked valid (you can see the global color table at the start) I decided to have a go at downloading it and viewing the GIF. Immediately I ran into trouble because the GIF was invalid. Looking at the code in a hex editor I immediately noticed that it was UTF-8 encoded. A quick run through UTF-8 decoding and the GIF was valid, but truncated. Unfortunately it's not very interesting when viewed in a web browser:
I wrote to the webmaster of the USAA web site, but have heard nothing back. Does anyone know if anyone else uses this technique?
PS This system measured my connection speed as 20Mbps. Which is about 5x too large.
The spider was simply requesting http://www.usaa.com/ which then goes through a sequence of complicated redirects. Here's the output from the excellent HTTP debugging tool HTTPFox:
GET 301 Redirect: https://www.usaa.com//
GET 301 Redirect: https://www.usaa.com/inet/ent_logon/ent_login_redirect.jsp
GET 301 Redirect: https://www.usaa.com/inet/ent_logon/Logon?redirectjsp=true
GET 200 https://www.usaa.com/inet/ent_logon/Logon?redirectjsp=true
POST 200 https://www.usaa.com/inet/ent_logon/Logon?redirectjsp=true
The critical page to watch is https://www.usaa.com/inet/ent_logon/Logon?redirectjsp=true. When loaded with no cookies it is served containing JavaScript code that captures information about the browser and posts it back to the server:
var originalUrl = "https://www.usaa.com/inet/ent_logon/Logon";
var queryString = "?redirectjsp%3Dtrue";
function submitForm() {
var obj = document.frmBrowserProfile;
obj.action = originalUrl ;
obj.action = obj.action + decodeURIComponent(queryString);
obj.bpsw.value = getScreenWidth();
obj.bpsh.value = getScreenHeight();
obj.bpsd.value = getColorDepth();
obj.bpbw.value = getClientWidth();
obj.bpbh.value = getClientHeight();
obj.bpap.value = getAcrobatVersion();
obj.bpfp.value = getFlashVersion();
obj.bpcs.value = getConnectionSpeed();
obj.submit();
}
The interesting part is the function getConnectionSpeed() which is written as follows:
function getConnectionSpeed()
{
var connectionSpeed = "999999999";
try
{
var datasize=10240;
var diffTimeMilliseconds = endTime - startTime;
if(diffTimeMilliseconds > 0)
{
var diffTimeSeconds = diffTimeMilliseconds/1000;
var bits = (datasize*8);
var kbits = bits/1024;
var connectionSpeed = kbits/(diffTimeSeconds);
}
}
catch (e) {} ;
return connectionSpeed;
}
And that relies on knowing a startTime and endTime. They is done by embedding a 10k GIF inside the web page, not by loading it, but by embedding it inside a JavaScript comment. The time is measured at the start and end of the GIF being loaded into the browser and the bandwidth determined from that.
Here's a shortened (and wrapped) version of the code:
<script language = "Javascript">
date = new Date();
startTime=date.getTime();
</script>
<script language = "Javascript">
/*GIF89a?*÷ÿÿÿÿ)))111999BBBJJJRRRZZZccckkksss
{{{????????????¥¥¥µµµ½½½ÆÆÆÎÎÎ{ssRJJ9111))Æ¥?
{sJc!ïÖÆR!?Jν½??Z)µk!µc?Jc½s!?R½kk9Jµk?R?Z!?
c?Rk{JsB?k9?c1cB?c?c?RR9B){Z!kJ?kZ9??R?sBcJ?k
?c¥s¥?9¥{!R9?{Rsc9?RkZ1cR)ZJ!?s!ƽ¥?k??BcR!µ?
?kkRcJ{Z?c?{B{k9sc1¥?9?Z¥?R??1ZJ??k??c¥?s?{R?
?B?{1{c??c??Z{sJ?B??9JB?{!ZJ¥s??RcZ!RJ¥?)?{{k
B9?s¥?Z?{9sk1cZ¥cµc??R¥R??B¥?J??B{s!kcRJZR??)
??!??!{s¥µµÆƽ???{{sssk½½Îνµµ¥RRJJJBÆÆ¥¥????
ÆÆ¥ccR??{½½?µµ??RRBÎÎ¥½½?µµ?JJ9¥¥{{{Z!!{??c¥¥
k??Rk¥¥c??RZZ1??Bss9BBZZ!??)kk!RR??)ss!??cc))
11??RZc!µ½???{ckJÆÎƵ½µ????¥?kskµ?????¥BBJ{{?
99Bkk?{{?ZZsss?ss?))9kk?cc?BBckk¥JJsZZ?cc¥kkµ
ccRR?cc½JJ?11cRRBB?99{!!JBB?))cJJµJJ½99?RRÖBB
µ))sBBÆ))?99½))?11Æ!!?))))Æ!!!!µ!!Æ!!Î!!Ö÷ÞÎ!
ùÿ,?*ÿÿýÛ?!@?U2$4¸
...
Æ;L@VáÀÂSØ?SHX༯?Aq ?âA@³-yRº`?ÔW85¾?ò*/
</script>
<script language = "Javascript">
date = new Date();
endTime=date.getTime();
</script>
Now the GIF data looked valid (you can see the global color table at the start) I decided to have a go at downloading it and viewing the GIF. Immediately I ran into trouble because the GIF was invalid. Looking at the code in a hex editor I immediately noticed that it was UTF-8 encoded. A quick run through UTF-8 decoding and the GIF was valid, but truncated. Unfortunately it's not very interesting when viewed in a web browser:
I wrote to the webmaster of the USAA web site, but have heard nothing back. Does anyone know if anyone else uses this technique?
PS This system measured my connection speed as 20Mbps. Which is about 5x too large.
Labels: pseudo-randomness
posted by John Graham-Cumming at 08:10 Permalink
6 comments
links to this post
