Which countries have the most beautiful women? (My deeply flawed analysis)

So, I happened upon the Wikipedia page about the Miss World pageant and noticed that it had a list of winners by country. For example, India has won Miss World 5 times. But, of course, India has a very large population so you'd expect it to be able to churn out a few beauties. So, to get a better idea here is a population adjusted list of countries that have won Miss World:

Country	Wins	Pop.	Wins/Pop.	Normalized
Bermuda	1	66163	0.0000151141876879827	100.00%
Iceland	3	316252	0.00000948610601672084	62.76%
Grenada	1	110000	0.00000909090909090909	60.15%
Guam	1	173456	0.00000576515081634536	38.14%
Jamaica	3	2651000	0.000001131648434553	7.49%
Trinidad and Tobago	1	1305000	0.000000766283524904215	5.07%
Sweden	3	9182927	0.000000326693221017656	2.16%
Puerto Rico	1	3994259	0.000000250359328225836	1.66%
Austria	2	8316487	0.000000240486157195941	1.59%
Ireland	1	4339000	0.000000230467849734962	1.52%
Finland	1	5308208	0.000000188387493481793	1.25%
Venezuela	5	28199822	0.000000177306083705067	1.17%
Israel	1	7282000	0.000000137324910738808	0.91%
Netherlands	2	16408557	0.000000121887622415548	0.81%
Dominican Republic	1	9760000	0.000000102459016393443	0.68%
Czech Republic	1	10381130	0.0000000963286270377117	0.64%
Australia	2	21290000	0.0000000939408172851104	0.62%
Greece	1	11216708	0.0000000891527175353054	0.59%
Peru	2	28674757	0.0000000697477575834383	0.46%
UK	4	60487300	0.0000000661295842267716	0.44%
Argentina	2	40301927	0.0000000496254186555397	0.33%
South Africa	2	43700000	0.000000045766590389016	0.30%
Poland	1	38518241	0.0000000259617255107781	0.17%
France	1	64473140	0.0000000155103350015216	0.10%
Turkey	1	70586256	0.0000000141670639111387	0.09%
Egypt	1	80335036	0.0000000124478689472424	0.08%
Germany	1	82210000	0.0000000121639703199124	0.08%
Russia	1	142008838	0.00000000704181524251329	0.05%
Nigeria	1	148000000	0.00000000675675675675676	0.04%
US	2	304072000	0.00000000657738956562919	0.04%
Brazil	1	186757608	0.00000000535453420457174	0.04%
India	5	1132446000	0.00000000441522156464856	0.03%
China	1	1321851888	0.000000000756514409124179	0.01%

So, far and away, the top three are Bermuda, Iceland and Grenada. Given that Bermuda is the winner, and a tax-haven, and has a sub-tropical climate... Hamilton here I come!

The Spammers' Compendium finds a new home

Shortly after I announced that I was getting out of anti-spam the folks at Virus Bulletin contacted me about taking over The Spammers' Compendium. I was delighted.

Today the transfer is complete and the new home is here. It will be maintained and updated by Virus Bulletin. Please send submissions to them.

Bookmark-based registration

Recently, I've been learning Ruby on Rails and I can never learn anything unless I build something with it. I also recently read Programming Collective Intelligence and had a desire to use some of those algorithms too.

I'll post more about the actual web site I created another time; currently it's in alpha form running on Heroku. The web site is used for naming babies, the initial alpha-release is the girls' names only site EmilyOrEmma? which uses "Hot or Not" style voting and manages to incorporate in one page the Levenshtein distance, Metaphone algorithm and both item-based and user-based filtering.



But my biggest bug bear with baby naming web sites is the need to create an account. You can browse baby names all you want, but as soon as you want to do something like add a name to a list of favourites you are forced into registration. Or you don't have to register but anything you do is ephemeral.

For my site, I came up with a better solution: bookmark-based registration. If you visit the site you'll see at the bottom of every page a link to bookmark. This link is unique to you. It contains your user id and a hash which I used to prevent forgery.



Bookmark this link and you can return to your recommendations, saved names, etc. any time.

Interesting real-world Apache Problem

I'm working with a large client who has a number of web servers behind a load balancer. This morning one Apache 1.3 had failed to come up on one of them. The client sends a SIGUSR1 to each Apache once an hour to force a graceful reload. This particular machine had operated correctly restarting Apache once per hour for 54 hours (since a recent reboot of the machine) and then died.

A quick look in the Apache error.log file showed the following:

module "mod_jk.c" could not be loaded, because the dynamic module limit was reached. Please increase DYNAMIC_MODULE_LIMIT and recompile.

Naturally I went looking for a problem with mod_jk which was the wrong place to look. Scrolling through the log file I noticed that every time Apache restarted we'd get the error:

Cannot remove module mod_include.c: not found in module list

This was where the real problem lay. A quick httpd -l showed that mod_include was compiled into the client's Apache and looking in the httpd.conf revealed that mod_include was also being loaded with LoadModule:

LoadModule includes_module modules/mod_include.so

When a module is both statically linked into Apache and dynamically loaded you run into a nasty problem: Apache doesn't complain when you start, but it will fail to unload the double loaded module on exit. So for every SIGUSR1 a single slot of the DYNAMIC_MODULE_LIMIT was used up. The default DYNAMIC_MODULE_LIMIT is 64 and with 10 real dynamic modules and a boot once per hour it took 54 hours to consume every slot in the module limit.

Removing the errorneous LoadModule fixed the problem.

Digg 3 Million

A quick update on my previous estimate of Digg users shows that, as predicted, Digg passed the 3 million user mark during March, 2008.



Comparing this estimated data and data from the Digg API shows that around 20% of the 3m accounts are not active. I speculated before that these accounts had been banned for spamming or other activities (that's around 600,000 bad accounts).

Growth appears to be the same as before adding around 150,000 accounts per month.

Juxtaposition

It's 3am and there's a crisis somewhere in the world

If you follow US politics then you'll know that Hillary Clinton has a couple of ads that start with a 3am phone call to the Whitehouse. The first ad was intended as a slam against Barack Obama implying that he didn't have the experience to deal with such a crisis. The second is going up against John McCain claiming he doesn't want to do anything about the housing finance problem in the US.

You know if it's 3am and there's a crisis in the world there's only one place and only one man to call.

CTU and Jack Bauer.

First of all, he's already up. 3am is nothing to Jack. Hillary and McCain both look like they could use the sleep and Obama looks like he gets his beauty sleep every night. So, Jack's ready to go before any of them.

As much as you might think Obama is David Palmer (safe pair of hands), he's more like a Wayne Palmer (a slick little fighter) and you know what that means: gets blown up within five minutes of being president and then pops a brain vein and the evil VP has to take over. The only good thing to say about Obama is that he would call Jack.

Now, McCain might look like a Bauer type with his military background and heroic time spent as a PoW. But here's the difference. McCain was 5 years as a PoW, no one came to get him out so he can't be that valuable. Also, if Jack had been held hostage in North Vietnam for 5 years there wouldn't be a North Vietnam now. Because you can bet the life of the next random CTU cast member the Jack would have (a) escaped and (b) annihilated everyone involved.

So, that leaves Hillary. She can't tell sniper fire from a little girl with flowers. In fact she reminds me more and more of the evil Vice Presidents that pop up in every 24 trying to take power from the real president. Hell, she's probably even got backing from Phillip Bauer.

Only one word of warning: make sure it's a man that calls Jack. If it's a woman he's bound to have been involved with her, she'll turn out to be a double agent or her father will be evil, and Jack'll be distracted.

If it has to be a woman, make sure it's Chloe.

Multi-route (email and phone) self-aware phishing

Today, I received the following email:

This communication was sent to safeguard your account against any
unauthorized activity.

Max Federal Credit Union is aware of new phishing e-mails
that are circulating. These e-mails request consumers to click
a link due to a compromise of a credit card account.

You should not respond to this message.

For your security we have deactivate your card.

How to activate your card

Call +1 (800)-xxx-9629

Our automated system allows you to quickly activate your card

Card activation will take approximately one minute to complete.


Of course, I don't have an account with Max Federal Credit Union and this is obviously a phish. Notice that the English is quite right:

"For your security we have deactivate your card." and "You should not respond to this message." doesn't make sense in context.

What's more interesting is that the message itself warns you about phishing emails and asks you to call an 800 number.

If you call the 800 number an electronic voice reminds you again to never give your PIN, password or SSN in email and then proceeds to ask you for the card number, PIN, expiry date and CVV2. The assumption is that you've been warned twice not to do something in email, so it's OK by phone.

It's painful to see the phisher use the existence of phishing as a way to phish.

Names: Boys vs. Girls

Using data from the 1990 US Census I was amazed to discover that 90% of the US male population has one of 1,219 first names, but 90% of the female population has one of 4,275. There are 3.5x as many female first names as male first names.

The top 10 male first names are: James, John, Robert, Michael, William, David, Richard, Charles, Joseph and Thomas (which account for 23.2% of the male population; 50% of the population have one of only 60 names).

The top 10 female first names are: Mary, Patricia, Linda, Barbara, Elizabeth, Maria, Susan, Margaret and Dorothy (which account for 10.7% of the female population; 50% of the population have on of 139 names).

You can also see that all the variety in names happens between the 80% and 90%. For males 80% of the population is covered by 27% of the names; for females 80% is covered by 19% of the names).



The large numbers of female names appears to be because there are lots of variants of female names compared to male names. A quick run through calculating the Levenshtein distance between names and selecting the 10 closest for each gives an average distance of:

Male: 2.62
Female: 2.01

So female names are more 'similar' than male names, hence the variety created by all these variants.

The other thing we can extract from this data is the prevalence of names beginning with certain letters and weight adjust based on the occurrence of each name.





Things are much more polarized when you look at trailing letters (for example, the trailing letter A is an almost sure sign that it's a woman; the opposite is true of D):





So combining the two it's possible to give a 'maleness' score (the blue part) to each final letter:

BOUTS: The Complete Song Parodies

Back when my site UseTheSource was a "blog" (this was late 90s/early naughties) I wrote a number of song parodies. Someone emailed me and asked where they were.

So, here are my complete song parodies from back in 2001:

March 7, 2001: "Candle in the Wind"

Yahoo! was in trouble with banner ad sales falling, profits disappearing and the then CEO, Tim Koogle, was off to spend more time with his family.

Goodbye Tim Koogle
Though I never Yahoo! at all
You had the faith, to sell banners
While those around you failed
They crashed into the deadpool
And they whispered into your brain
You need to begin to charge
For things you give for free

And it seems to me you lived your life
Like a candle in the wind
Never knowing what to add next
To your list of links
And I would have liked to have told you
But I was just a geek
Your web brand burned out long before
Your stock price ever did

Jerry Yang was tough
The toughest boss you ever had
Softbank created a superstar
And pain was the price you paid
As the whole web died
Oh CNET still hounded you
All they had to say
Was that Google was the site to use

[Repeat chorus]

Goodbye Tim Koogle
From the young man on the DSL link
Who sees you as something more successful
More than just our long lost CEO

[Repeat chorus]

March 22, 2001: "Don't Cry for Me Argentina"

Steve Jobs was back at Apple, the blue iMac was out, Microsoft was in big anti-trust trouble and had just invested $100m in Apple, Steve had bought out NeXT. But the future wasn't yet assured:

This won't be easy,
you'll think it's strange.
When we try to explain what we need
that we now need your help
after all that we've said.

You won't believe us.
All you'll see is Apple you once knew,
although we've crashed down in the dumps
begging for Microsoft cash.

It didn't have to happen.
We should have won
Better software and patents than Bill
Looking down on Windows,
staying far from Sun.
So we chose NeXT.
Running aground, trying computers in blue.
But nothing revived us at all.
You never expected it too.

Don't cry for us, William H. Gates.
The truth is we're dead without you.
We need your dollars
We need Mac Office
You need a rival, for your survival.

And as for Fortune,
and as for Time,
We never invited them in
though it appeared, to the world,
they were all Steve desired.
Even Adobe,
they're making solutions for Windows right now
The answer was here all the time.
We need you, and hope you need us.

[chorus]

Have we begged too much?
There's nothing more we can think of to say to you.
But all you have to do,
is look at us to know,
we're through without you.

April 1, 2001: "I Just Called To Say I Love You"

In the midst of the crash, .coms were going out of business like crazy:

No IPO to celebrate
No friends and family stocks and shares to give away
No big opening
No first day ping
In fact here's just another ordinary day

No Aeron chair
No onsite chef
No working Saturday until the site is done
But what it is, is something blue
Made up of these few words that I must say to you

We just failed to get more funding
We just failed to keep our doors open
We just failed to get more funding
And we need it just to avoid bankruptcy

No free massage
No free soda
No caffeine trip to keep us working every night
No dry cleaning
No stock option
Not even time for us to pack our things and leave

No beanbag room
No Maui trip
No giving thanks to all that NASDAQ did for us
But what it is, though old so new
Grab what you can before your jobs right here are through

[chorus]

[chorus]

April 6, 2001: "Uptown Girl"

Ah, to be in love with a marketing .com girl:

.com girl
She's been living in her .com world
I bet she never had a software guy
I bet her mama never told her why
I'm gonna try for a .com girl
She's been living in her wide web world
As long as anyone in marketing can
And now she's looking for a comp. sci. man
That's what I am

And when she knows what
She wants from her time
And when she wakes up
And makes up her mind

She'll see I'm not a nerd
Just because
I'm in love with a .com girl
You know I've seen her in her online world
She's getting tired of her high tech toys
And all her presents from her VC boys
She's got a choice
.com girl
You know I can't afford to buy her a Porsche
But maybe someday when my stock cashes in
She'll understand what kind of guy I've been
And then I'll win

And when she's walking
She's using her Nokia
And when she's talking
She'll say that she's mine

She'll say I'm not a nerd
Just because
I'm in love
With a .com girl
She's been living in her latte world
As long as anyone in marketing can
And now she's looking for a comp. sci. man
That's what I am
.com girl
She's my .com girl
You know I'm in love
With a .com girl

April 13, 2001: "Gangsta's Paradise"

Linus Torvalds was the flavor of the day as one of the thorn's in Microsoft's side:

As I drive through the Valley of the Silicon Dream
I take a look at my life and realize there's nothing left
'Cause I've been coding and debuggin' so long
That even my manager thinks that my mind has gone
But I ain't never crossed a man that didn't deserve it
Linus treated like a punk, ya know that's unheard of
Ya better watch how ya postin'
And what ya codin'
Or you Dr Tanenbaum'll be lined in chalk
I really hate Minix and FreeBSD
As they croak, I see myself in the pistol smoke
Fool, I'm the kinda hacker script kiddies wanna be like
On the Net in the night, writin' layers of the core code

CHORUS:
Been spending most our lives living in a Windows paradise
Been spending most our lives living in a Windows paradise
Keep spending most our lives living in Bill Gates' paradise
Keep spending most our lives living in Bill Gates' paradise

Look at the situation they got me facing
I can't live a normal life, I was raised on the PC
So I gotta be down with the kernel team
Too much crazy Usenet posting got me chasing dreams
I'm a educated fool with Posix on my mind
Speak Swedish in my home and English on the phone
I'm a loc'd out hacker, wrote my life story
And my homies is down so don't arouse my anger
Fool, death ain't nothin' but a heart beat away
I'm livin' life, do or die, what can I say?
I'm 28 now, but will I ever see 29?
The way things is going, I don't know

Tell me why are we so blind to see
That Microsoft's a monopoly

CHORUS
CHORUS

Tell me why are we so blind to see
That Microsoft's a monopoly

Tell me why are we so blind to see
That Microsoft's a monopoly

April 18, 2001: "Copacabana"

Carly Fiorina was fighting for her life as she tried to merge HP and Compaq with Walter Hewlett attempting a proxy fight to stop her in the name of the family:

Her name was Carly, she was a VP
With Lucent and AT&T and a degree from MIT
She went to HP and wowed the board room
And while she tried to be a star, sometimes went a bit too far
And then September 4, Compaq became the score
They were failing and needed each other
Leaning drunks galore!

At the HP, HP/Com-pa-q
The merger that upset the family
At the HP, HP/Com-pa-q
David and William were always the fashion
At the HP... they ran the show

His name was Walter, his dad was famous
He wasn't present for the board, but he wouldn't be ignored
And what she pro-posed, "Dad would've hated"
Then Walter went a bit too far, "Carly: time for au revoir!"
And then the insults flew and careers were smashed in two
There were ads and a lot of bankers, but just who screwed who?

At the HP, HP/Com-pa-q
The merger that upset the family
At the HP, HP/Com-pa-q
David and Walter are today the fashion
At the HP... they run the show

Her name is Carly, she was CEO
But that was 30 weeks ago, when she used to run show
Now she's a VC, but that's our Carly
Still in the suit she used to wear, new blonde highlights in her hair
She sits there so refined, and drinks to Walter's health
She lost her job and she lost the proxy, now she enjoy's her wealth!

At the HP, well just the HP
The toughest job belongs to Walter
At the HP, well just the HP
William and David were always the fashion
At the HP, don't buy the stock...

If you find these funny and can sing... feel free to set them to music and give me a laugh.