Commentary on commentaries

During my high school days, one of the primary books that i had to cover is William Shakespeare's Macbeth. At that time, one couldn't really grasp the prowess of his words and phrasing that captured every essence in life. I guess you really need life's experiences to complement such classic literature.

I can still easily recite some of the soliloquys from the main character. One of my favorite quotes from Macbeth is in Act V, Scene V, where Macbeth, upon hearing the death of his wife, cited:

"Out, out, brief candle! Life's but a walking shadow, a poor player that struts and frets his hour upon the stage and then is heard no more: it is a tale told by an idiot, full of sound and fury, signifying nothing."

The last line has multiple layers of meaning to it. The layer that i'm focusing on this post is about the comment that Shakespeare made on what constitutes good drama. I believe that the comment applies to sports, and other areas as well.

Being a boy in his teenager years in Singapore, chances are that he would be playing football (aka Soccer in USA), and would have been watching the Big League Soccer from the UK. At that time, the telecast would be a straightforward showing of the football match, with minimal commentary. Similarly, the football World Cup matches would be shown on TV right at the minute the matches started in the host countries. You'd watch the game, have a break during half time, and finish the game right at the final whistle.

Nowadays, it is not uncommon to see a pre-game show that lasts for an hour. Sometimes for big events like the World Cup final, the pre-game show might be three hours for a 90 minute match! A big team of former players, commentators, "experts" and other guests will be sharing their points of view with one another.

Similarly, during the actual match, the commentators will start to cite tons of metrics, for example:

1) how many assists each player make this season

2) how many tackles (successful/failed)

3) how much playing time each player has

4) so on and so forth....

I wonder how all these metrics actually matter. At the end of the day, isn't the one important data point is whether one team scores more goals than the other? Team A can make 1000 assists, has 100 shots at goal, make 100 successful tackles, and ran 10000 yards during the 90 minutes, but if Team B can make their 1 shot at goal count, nothing else matter. The top scorer in the World Cup 2006 was Germany's Miroslav Klose, who had a two goal lead against the next set of top scorers, but the final was between Italy and France.

What all the pre-game/post-game shows and the in-game commentaries have done is to create a huge and profitable market in terms of allowing these non-playing "experts" to give their two cents' worth of opinion in as many media channels as they can.

I recalled that there was a series in probably early 2002 that had two sets of commentators that were spouting different "data" on the same players in the same season. One set would say that Zola passed 212 times in the month of August, while another set said it was 195 times. It became a farce when they started arguing about the definition of a pass from their respective point of view!

Is there any real value added to the game? No. The unnecessary hype of such useless metrics not only do not add value to the game, but rather, shifts people's focus off the most important goal, which is, who score more goals!

This also created some false sense of authority on these "experts". Again, the ones that gained from these "sound and fury" are the ones that go around spreading sound and fury. I pity the players that are doing the actual sports. At least in terms of football players, most top league players are getting rich enough that they can afford to ignore this kind of distraction.

Technorati Tags: Commentary , Shakespeare , Football , Soccer , World Cup , Macbeth

Optimizing for metrics

While waiting for Sharon yesterday, i was at the company's in-house library and read from cover to cover two to three small but good books. The topics range from various software development models, Mathematical Puzzles (gosh, how i missed doing the math and logic puzzles from Martin Gardner when i was in high school) and one on good management and leadership skills on a quality team!

The first book (Smart and Gets Things Done) that i just completed in about 20 mins was a book by Joel. Some of the chapters do not apply to me since i'm not a hiring manager, but the concise but very detailed chapters are great to incite ideas in my brain. I can see someone already shivering from afar.

An example that he mentioned on Starbucks vs other local cafes was just something that i personally observed recently. Having the chance to visit two similar Mexican fast food places, Chipotle and Qdoba, that have practically the same setup in terms of layout and menu options. They even had the same number of people queuing up for orders but somehow, one had a very quick flow while the other was stuttering along.

The only thing different is that Chipotle has a very optimized process from the point that the customer orders the food to the point where the customer pays for the food. The order is only taken once, and the necessary information is passed down the "service line" in an effective manner. This is something that Qdoba didn't do. First the customer give his order to the first "server", and then to the one that handles the condiments, and then to the cashier eventually.

Three times vs one time. Hmm.. So where's the secret trick that Starbucks and Chipotle do that the others don't? The person taking the order WRITES down all the necessary information in acronyms on the wrapper/cup, and basing on the provided information, the rest of the service line can continue on without impacting the customer.

I wonder why that there's such a big difference in the way the two sets of examples are doing processes that are inherently the same?

One way of looking at it is perhaps the way that the organizations are measuring their staff. What is the critical metric that they should focus on? Is it the eventual customer satisfaction, or the measurement of how long each member along the service line take?

That line of thought link the first book to the book with the mathematical puzzles that i just couldn't remember the title for. A puzzle that was detailed is the well known Prisoner's Dilemma. In this game, as in all game theory, the only concern of each individual player ("prisoner") is maximizing his/her own payoff, without any concern for the other player's payoff.

Since each member along the service line is measured for his own efficiency, the quicker he can move the customer down the line the more rewards he will get. He will not care for whether the downstream gets the necessary information to work with, or worse, whether the customer is even being frustrated by the repeated queries on the same order again.

Would i blame the folks on the service line? No. If their management gives them such a metric for measurement, chances are that the folks will find ways to optimize for the specific metric that gets rewarded, without actually achieving the most significant outcome that is desired. In this case, i believe that the management might want happy customers that will be glad to come back for future purchases, but will they get such repeated purchases? I know that i won't want to spend time in Qdoba. If i want the same kind of food, i'll go to the more effective and efficient Chipotle.

More book reviews in the future. I promise a more regular update on this blog. 8).

Technorati tags: book review, starbucks, fastfood, chipotle, qdoba, joel, notes

Real life testing

So, what is a test? Among many other definitions, it's "achieve a certain score or rating on a test". Most tests that one will do is constrained to a certain subject (for example, Elementary English), or even just a certain area within a specific subject (Estimating the Airspeed Velocity of an Unladen Swallow in Physics).

Why should we have a limit in the areas that we're tested on? For one, it's to protect the folks that are taking the test. For example, there's no way that an English teacher, while conducting a test in Elementary English, will fail a student in the test just because he has no knowledge of Klingon language.

Another example that i can think of is that one will not fail his Elementary English test because he has a number of errors in his Physics test.

Usually, the passing standard is usually agreed upon before one takes a test. One doesn't necessary get to "agree" on the standard, but at least the standard is known. For example, the passing grade for the driving test here in Washington is 80 out of 100. You don't get to agree on it, but you know it well before you take your driving test.

A controlled environment is quite common in tests as well. You don't get to do your Elementary English test in a Indy 500 racing course; neither do you do your driving test in a high school car park (i hope not!).

Objective testing vs subjective testing is always something that can be argued till kingdom come. Even in our primary example of Elementary English, you can have an objective test using multiple choice questions vs an open and subjective composition test.

Subjective testing scores are further skewed when the examiner has a goal in mind before evaluating the test entries. My high school English teacher usually score a "A" grade for me and a "C" grade for my classmate, basing on the fact that he was one of the students that was weak in English while i was always in the top two for English in the high school. So, we decided to play a prank on the English (he was an expatriate from Norwich, England) teacher and we wrote our usual compositions, gave each other our original work, and copied each other's work 100%.

At the end, I still received a "A" for my friend's usually "C" work, while my friend got a "C" for my usually "A" work! That's not totally scientific, but hey.. that proved a point. If you have a predetermined goal in mind, whatever that you are testing for will be affected.

So unless you are perfect, errors will be found (perhaps a typo, a grammatical error, drove through a red light etc) and you might fail the test. One thing that you shouldn't be justifying will be the fact that you drove through the red light because the light was too dim, or your pencil suddenly made you do a typo. As another well known antivirus researcher has mentioned, "No point in looking at the color of my shirt and scoring my application based on that.". You also shouldn't expect people to mark your test based on irrelevant, or worse, constantly changing baselines during a test.

Most importantly, if you fail, you are held accountable for the failure. You either have to retake the test, take another subject, or fail the whole course altogether. Your actions will be based on your prior actions. As easy as that. Nothing of those comments such as "Wow.. your test is very difficult. Are you sure this is meant for Elementary grades instead of High School levels?", "Hmm.. your questions made me look bad. Let's take these questions out and give me some easier ones to handle.".

So what should you do if you fail? I hope that you don't go crying to your parents. They shouldn't help you to argue with the instructors for a passing grade. They should help you to find out why you have failed, and help you to improve. If you lose the World Series, you don't go crying to your coach and ask the Major League to replay the series again. You lose the series, you figure it out, and move on. Similarly, Zidane lost the World Cup for France. He cried. Even though it was found out later that the Italian defender Marco Materazzi did insult his family, he (and France) took the defeat in a sporting manner. Did he ask for a replay? No.

That's how real life from my life in Singapore has shown me. I don't think the examples above really works that way here now.

What do you think?

Technorati tags: testing, exams, Singapore, observations