RUMINATIONS

getting mysql-python installed on MAMP

2012-01-27T15:01:00.000-08:00

I wanted to try out selenium to setup some test automation scripts. So i proceeded to get a python virtual environment setup where i could monkey around a bit. Its a brilliant idea to use this to test out something before deploying or deciding to implement it company wide.

install virtualenv ( http://pypi.python.org/pypi/virtualenv ). its just a single file.
So I setup a virtual environment for testing selenium :-

python virtualenv.py testselenium

    cd testselenium

    source ./bin/activate

gives you a similar looking prompt by default:
      (testselenium)Ajoy:testselenium ajoy$

now install selenium through pip:
      pip install selenium

this should usually go thru smoothly
In case, i also wanted to connect to mysql to pull in db stuff,
So I needed to install mysql-python:
      pip install mysql-python

but it complained that it cannot find 'mysql_config'... grr .... need to setup the paths

in my case, it was installed in : /Applications/MAMP/Library/bin/
      export PATH=/Applications/MAMP/Library/bin/:$PATH

try again :
      pip install mysql-python

now it complained that "ld: library not found for -lmysqlclient_r" .... Grrrr....

need to find where the mysql libraries were installed,
     export PATH=/Applications/MAMP/Library/lib/mysql/:$PATH

and setup DYLD_LIBRARY_PATH
macports mysql debug client libraries:
     export DYLD_LIBRARY_PATH=/opt/local/lib/mysql5/mysql/

it still did not work, then i came across this blog ( fixing-ld-library-not-found-for-lmysqlclient_r-during-gem-install-mysql2-on-osx-w-mamp ).

so now i created symbolic links as adviced on the site :-

   cd /Applications/MAMP/Library/lib/mysql/
   ln -s  ibmysqlclient.la ibmysqlclient.a
   ln -s libmysqlclient_r.la libmysqlclient_r.a
   ln -s libndbclient.la libndbclient.a

now, everything worked fine :) *phew*

Google Analytics Gotchas

2011-11-01T22:34:00.000-07:00

Having built quite a number of websites over my career, there is one feature that I always end up implementing right at the very beginning, ... Web Analytics.

There is are several ways of going about it. Analysing apache logs, capturing session data are all low level ways of looking at the data. But when it comes to ease of use and powerful reports, nothing beats google analytics for the right combination of price (Free!) and features. With the gradual rollout of real-time analytics, it becomes even more powerful.

Google Analytics is a wonderful free tool every website owner should use. Getting setup with google analytics is incredibly easy. The official google analytics site has very clear and succinct instructions:- http://code.google.com/apis/analytics/docs/tracking/asyncTracking.html

This basic setup should give you reports about visits and pageviews and track your site's popularity over time. Other interesting metrics to look for is bounce rates, pages getting the most pageviews and how the users are landing on your site ( also known as referrals). That's a good place to start on how to optimize your site.

You might find that having a distinct URL for each specific page on your site helps with tracking its usage seperately. This will be tricky to do if your main page is being reloaded with new content using Ajax. There are two ways to solve this - Virtual PageViews and Event Tracking.

Virtual Pageviews
Virtual Pageviews are very simple. You force google analytics into thinking that a pageview has occured using a similar snippet of code :-

_gaq.push(['_trackPageview', '/module/specific_page']);

This will be added as a pageview in your report.

Event Tracking
Event tracking is a more versatile solution. The code tells google analytics that a particular event has occured. It is upto us to interpret and provide attributes of that event. Snippet of code :-

<a onclick="_gaq.push(['_trackEvent', 'category', 'action', 'opt_label', opt_value]);">click mea>

The number of times each event occured and pages in which it occured can each be tallied in the report.

Event tracking also allows us to capture more intricate details of a pageview. It is normally used to capture user interactions on the page that may or may not result in new content. Very useful when it comes to determine whether that button that you added, how often is it being used. Or is it just a distraction for the user.

Website Profiles and Filters

MySQL Replication on EC2

2011-06-06T12:38:00.000-07:00

I am not a DBA, but being in a 3-person start-up makes you a lot of things, at least temporarily. I am not a writer either, but I am hoping that I can share what I have learned and hope somebody else finds it useful.

While running a commercial web application, two things are of at-most importance, reliability and performance. For these two very important parameters, the most common bottleneck tends to be the database. This leads me to the topic of MySQL Replication.

Assuming you have two machines running MySQL Database servers, MySQL replication can be done in two ways, Master-Slave and Master-Master combinations.

Master-Slave
Master-slave combination is used usually for performance reasons. The writes go to the master db server, which are then replicated on the slave db server. The slave can then be used for slower queries and reads, freeing the master for more important work.

Master-Master
Master-Master combination in addition to performance, are also used for reliability reasons. The writes generally go to one of the masters, also known as the primary master (which is pre-determined), and is replicated to the secondary master. The secondary master serves as a stand-by, in case something goes wrong with the primary master. The secondary master can then pick-up the writes (thus becoming the primary), which are then replicated to the old primary master, (now the secondary) when it is brought back up. This provides redundancy. But there are some important caveats to keep in mind, which I will come to later. Since this setup is more complex, I will take you through this below.

EC2
Setting up this replication on EC2 is a little bit tricky. But I will guide you along so that you know exactly what to do. There are quite a number of steps, but each one is fairly easy to do, just need to be careful.
Machines on EC2 have what are known as a public DNS and a private DNS. if you have logged in to the machines, then you have most definitely used their primary DNS. You can obtain these names from the aws console.

For MySQL replication purposes, we will need the private DNS/private IP Address.

Here is the setup that I had :-

Two 'm1.large' ec2 instances
Ubuntu 10.6
MySql 5.1

Detailed Steps

Now login to one of the instances, which is your primary ( hereby called server P ). It doesn't matter which one is primary, but if you have already been using one of them, then that is your primary.
Now login to the MySQL database instance and create a new user. This user will be used by the secondary master ( hereby called server S ) to connect and obtain updates. :-

> USE mysql;

> INSERT INTO user (Host, User, Password, Select_priv, Reload_priv, Super_priv, Repl_slave_priv) VALUES ('%', 'slave_username', password('slave_password'), 'Y', 'Y', 'Y', 'Y');

> FLUSH PRIVILEGES;

Find out where your mysql configuration file is located ( my.cnf ). In my case, it was located at /etc/mysql/my.cnf. There should be a 'conf.d' folder in the same level.
Create a new file called replicate.cnf in /etc/mysql/conf.d/ folder. This file will contain the customizations for replications, such that we don't disturb 'my.cnf' for now.
Insert the following :-
[mysqld]
server-id = 1 # for server P
replicate-same-server-id = 0
auto-increment-increment = 2
auto-increment-offset = 1 # set differently for server S

master-host = ip-11-111-1-111.us-west-1.compute.internal # private DNS of server S
master-user = slave_username
master-password = password_for_slave_user
master-connect-retry = 60
replicate-do-db = database_to_replicate

log-bin = /var/log/mysql/mysql-bin.log
binlog-do-db = database_to_replicate
binlog-ignore-db=mysql
log = /var/lib/mysql/mysql.log

relay-log = /var/log/mysql/mysql-relay-bin
relay-log-index = /var/log/mysql/mysql-relay-bin.index
relay-log-info-file = /var/log/mysql/mysql-relay-log.info
Repeat steps 2 - 7 for server S.
For the replicate.cnf for server S, insert the following :-
[mysqld]
server-id = 2 # for server S
replicate-same-server-id = 0
auto-increment-increment = 2
auto-increment-offset = 2 # set differently for server P

master-host = ip-22-222-2-222.us-west-1.compute.internal # private DNS of server P
master-user = slave_username
master-password = password_for_slave_user
master-connect-retry = 60
replicate-do-db = database_to_replicate

log-bin = /var/log/mysql/mysql-bin.log
binlog-do-db = database_to_replicate
binlog-ignore-db=mysql
log = /var/lib/mysql/mysql.log

relay-log = /var/log/mysql/mysql-relay-bin
relay-log-index = /var/log/mysql/mysql-relay-bin.index
relay-log-info-file = /var/log/mysql/mysql-relay-log.info
With me so far? Ok. Make sure '/var/log/mysql/mysql' specified in the cnf actually exists. For the initial stage, you need to manually make sure the data is in synch. This can be done by doing a mysqldump of the database on the primary, and ( scp ) copying the resultant file over to the secondary and running it there. Here is a link to help you out.
The mysql default configuration prevents the database from being accessed from different machines. Copy the default my.cnf into my.cnf.bak. Then edit the /etc/mysql/my.cnf file on both servers, and comment out the
#bind-address = 127.0.0.1
The last thing needed to set up replication is the synchronization of the servers. On the mysql command-line on server P, enter the following :-

mysql> show master status;
+------------------+----------+--------------+------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+------------------+----------+--------------+------------+
| mysql-bin.000006 | 106 | fitz | mysql |
+------------------+----------+--------------+------------+

Keep track of those values as you need this on server S.
On server S, mysql command-line:-

mysql> stop;
mysql> CHANGE MASTER TO MASTER_HOST='', MASTER_USER='', MASTER_PASSWORD='', MASTER_LOG_FILE='< Eg. mysql-bin.000006 >', MASTER_LOG_POS=< Eg. 106 > ;
mysql> slave start;
mysql> show slave status;

This should show that the slave is running or waiting for event from master.

15. Repeat step 14 for server P, make sure you plugin all the right values.

Thats it!
Test to make sure the changes made on one database is being propagated to the other. You can also do a tail on the logs :-

tail -f /var/lib/mysql/mysql.log

Enjoy!

As a manager, how do you motivate your employees?

2010-09-30T21:03:00.000-07:00

Some people refuse to be a manager. They are scared that managing others places a huge responsibility on their shoulders and distracts them from work that they need to get done.
As a recent manager, I can see where that fear comes from. At the same time if done properly, it is rewarding as well. Working as a team, you can accomplish a lot more than what you can get done as an individual.

Setting up OpenVPN (windows client, linux server)

2009-02-26T13:13:00.000-08:00

I was working on certain pet projects of mine and I realized that i might have to access my server machine from remote locations. I could always ssh by opening up a port, but I felt VPN would be more secure, with the added benefit of more security over free wireless points in cafes or other public locations. I have to warn you that the following texts are a bit lengthy and detailed, but when you start doing it, its fairly easy.

The free and seemingly well regarded option for vpn was openVPN.
I had installed xubuntu on my desktop/server box. You need to have openssh and openssl installed before attempting to install openVPN.

installing openVPN on ubuntu was fairly easy

sudo apt-get install openvpn

I installed the stable version openVPN 2. The main configuration files are located in

/usr/share/doc/openvpn

The "easy-rsa" folder was under the "examples" folder here. I followed the instructions as specified in the "openVPN Howto".

There are 2 types of VPNs you can setup :- Ethernet Bridge and Router based. Ethernet bridge is used when you want the client machine to be a part of the private subnet and allows for multi-casting (which is required by most LAN games). At the same time, routing is recommended for other purposes, so went with the router-based option.

Copy over the "easy-rsa" folder over to

/etc/openvpn/

Edit the "vars" file in the easy-rsa folder.
Make the following changes :-

export D=/etc/openvpn/easy-rsa/2.0

export KEY_COUNTRY=US
export KEY_PROVINCE=CA
export KEY_CITY=SanFrancisco
export KEY_ORG="My-OpenVPN"
export KEY_EMAIL="mine@example.com

Now run

. ./vars or source ./vars

then

./clean-all

The next command creates your certificate authority (CA) using the parameters you just set, you should just add Common Name, I used OpenVPN-CA. For this step you'll need OpenSSL.

Now we need to create the certificate :-

./build-ca

Now let's create the keys, first the server:

./build-key-server server

This is important. When build-key-server asks for Common Name write server, the same parameter you provided to the command. Also you'll need to answer yes to these two questions: Sign the certificate? [y/n] and 1 out of 1 certificate requests certified, commit? [y/n].

Now the key for the client:

./build-key client1

Use client1 as Common Name, the same parameter you used above for build-key.

Now let's create Diffie Hellman parameters:

./build-dh

There you are! Now you should have a new directory with your certificates and keys: /etc/openvpn/easy-rsa/keys. To configure your first client copy these files from server to client:

ca.crt (required for all clients)
client1.crt (specific to each client)
client1.key (specific to each client)

Now that the keys are set, modify the server openvpn config file as specified in the openVPN Howto.

For the client, i had used a windows Vista machine. Download the windows installer exe from the openVPN.net site and install everything.
Create a "keys" folder under "C:\Program Files\OpenVPN\easy-rsa". Copy over the client specific files over from the server where you had generated them (FYI: this is has to be done securely!).

Copy the client.ovpn file from C:\Program Files\OpenVPN\sample-config into the "keys" folder.

Modify the "client.ovpn" file to be consistent with the server. Modify the following directives:-

"dev-node" (name of the TAP Win32 Adapter)
"remote"
"ca"
"crt"
"key"
"ns-cert-type server" ( i enabled this, since it helps prevent certain kinds of attacks)
"tls-auth" (if you had created this key on the server earlier, read "Hardening OpenVPN Security" in the openVPN Howto section).

Now, you are ready to run the VPNs :-)

Start by running the server. it can be invoked as :-

openvpn server.conf ( i did this on the linux box )

it should throw up a bunch of messages ending in "Initialization Sequence Completed".

Run the client also from the command prompt ( i didnt install the GUI ) in a similar fashion:-

openvpn client.ovpn

Visualizing Data using a Heat Map

2008-06-12T23:35:00.000-07:00

A very common data visualization problem that most analysts come across is how to show variations in an attribute across two dimensions.

For Example, say you own an internet commerce site. After you have crossed the threshold of a few hundred users an hour, you think... hmmm... maybe i should show some featured products/good deals that i want new users to see. You start to show products that you think appeals to your audience. Great. Things are working out, but after a while, your site has again hit a plateau.

You start to wonder. How else to improve the site? The most important thing to do is to make sure that the site has sufficient tracking code embedded in it, so that everything and anything a user does on the site can be recorded and analyzed later. This information is invaluable. Otherwise it is very hard to do any sort of optimization. Once you have the data. you could analyze the buying patterns of my users. Look at what gets sold when, or what is the geographical distribution of products bought. There are lots of ways, the data can be analyzed.

The problem these days is not that there is no data, there is too much of it and making sense of it can be made easier with good visualization techniques. Tracking a parameter across two dimensions is quite easy using a simple 2D graph:-

Tracking an attribute across two dimensions when there is two sets (or series) of data is also possible and common:-

Here we are using color to distinguish the variations of the attribute between the two dimensions with two series.
But what if instead of two series, you need need to track 20 or 30 series, it can quickly become quite overwhelming and the visualization is useless in this format. The different series form a dimension of their adding a third dimension to the fray.
For example, take a look at this graph:-

Even with 7 series, the graph is confusing. So imagine 20 or more. One option might be to draw a 3d graph or surface, but you need the dimensions to be continuous and numerical. If one of the dimensions are discrete and categorical, the 3D graph won't make much sense.

Thats were the heat map comes in. It allows the presentation of variations of an attribute across two discrete categorical dimensions. If as in the example presented earlier, you are tracking the sales of products during the time of day across different categories of products. The heat map allows you to present the variation in sales as different gradations of colors. The categories can be presented along the x-axis and the time of day along the y-axis or vice-versa. The sales for each intersection of the dimensions can be used to determine the color of the cell.

Stupid, Smart and then there is too Smart!

2008-01-23T22:37:00.000-08:00

There is such a thing as being too smart. Everybody knows that. When someone gives a smart-Alec remark, you accuse the person as being too smart. As there are people who can to be too smart, so does software.

Developers start to make their software too smart usually when they run out of ideas for new features. Sometimes "it" also happens when developers think they are smarter than their software users. Researchers love this stuff, it has optimization and machine-learning written all over it, countless papers in conferences can be churned out, to the eventual detriment of the product.

Let me give you an example of what i mean by being too smart.

Say, I went to the local appliances stoer and bought the top-of-the-line, hyper-expensive, super-hyped coffee making machine, that brews the coffee, pours creamers and sugar as needed, and out comes your steaming cup of coffee just right as you always love it, .... or do you....

Assuming that I figured out how to operate the machine, I drink my first coffee with dark-roast coffee beans, 1 tablespoon of cream and 1 tablespoon of sugar. It turns out perfect. I love it! Second time, I do the same thing. I keep doing the exact same thing for a few weeks. Now this is a smart machine, after a while it learns, ...yes, it learns what I like, after all its smart. So next time I start the machine, it "thinks" it knows what I need and automatically makes the coffee that I always made. This is fine and dandy, until one fine day, i felt like having coffee without the creamer. But whatever I press, out comes coffee with 1 tablespoon of cream and 1 tablespoon of sugar! Hmmm, let me look at that manual again.....

Now the machine has become too smart.

Usability experts will tell you that users like to have a sense of control. You might have the smartest algorithm that can read people's minds, but unless you want people to be intimidated by your product, you have to provide them a sense of control. This can be done in a number of ways. In the coffee machine above, provide instant feedback on the choices its made and easy ways to correct it if needed.

The same principles apply to software that interacts with users.
Jacob Nielsen lists the "Ten Heuristics for User Interface Design":-

Visibility of system status

The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.

Match between system and the real world

The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.

User control and freedom

Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.

Consistency and standards

Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.

Error prevention

Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.

Recognition rather than recall

Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.

Flexibility and efficiency of use

Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.

Aesthetic and minimalist design

Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.

Help users recognize, diagnose, and recover from errors

Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.

Help and documentation

Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.

Web UI - "Light"s and "Flex"ibility

2007-06-02T17:52:00.000-07:00

A lot of new stuff is happening in the web UI field. Microsoft's recently introduced "Silverlight" and Adobe's recently open-sourced Flex has generated some excitement in the field. Though Adobe's Flash technology remains the overall leader in highly interactive apps, they are slow to load and Adobe's software has become very expensive. After reading recent comments by their CEO, mentioned in a blog: "our customer is not typically price sensitive", I dont feel comfortable investing in a technology that can later hold me hostage.

For this reason alone, i am thankful to microsoft for their Silverlight technology. Competition is what Adobe needs. As many of Front-End Engineers might know, getting the javascript and DHTML to work as you want on all the myriad browsers is a nightmare. Each browser has its own quirks in the way they have implemented the javascript and CSS, which turns the design of the front-end into a shamble of code hacked to get around various bugs.

There are two ways programmers have got around this problem. One way is to use various javascript libraries, which hide the incompatibilities within the library, and expose a standard API. The most prominent of which are the Yahoo! UI library and Dojo. Each library has its strengths and weaknesses..... the alternative was to use Flash. A third option I did briefly look at, is an open source solution known as open lazslo. The idea was right, but it still didnt work like i wanted to. Open Lazslo allowed you to define your interface and interactions using a combination of xml and javascript. It then generates Flash or in the latest version, DHTML, depending on the browser agent.

Both Adobe and Microsoft are trying to expand their reach, in opposite directions. Microsoft trying to move into the online universe while Adobe, with their Apollo platform and media player trying to move into the desktop space.

Oh, by the way, i am really excited about Microsoft surface :-)
Refer to a link in my earlier post ( ted talk ), a demo of a multi-touch point screen, which is pressure-sensitive, while microsoft's technology uses 5 cameras, and is visually sensitive.

Mobile Phone Interfaces

2007-01-25T00:37:00.000-08:00

Speaking of Mobile phone interfaces, saw this recently among the many CES technologies that were showcased recently :-
http://news.com.com/1606-2-6150139.html?tag=st.rb

This is an innovative way to solve the problem of having to press a key multiple times to cycle through 1 of 3 alphabets in a normal phone key.

its simple, and thats why i think its a good design.

The Apple iPhone has been creating waves with its new multi-touch point interface. It will be a while before it hits mainstream computer interfaces. But the technology already existed a while back and will revolutionize our interaction with the computer.

Other than the one tom cruise uses in minority report, i can live with this :-
http://www.ted.com/tedtalks/tedtalksplayer.cfm?key=j_han

Wish i had one of these :-)

My Experiences with Mobile Technology

2006-12-06T00:06:00.000-08:00

I read articles recently about how the new windows Vista Operating System, will make the PC the center of everything in your homes. In American homes out-fitted with 7.1 home theatre systems, with hi-def video content and almost infinite broadband, that might be a reasonable assumption. But i made a recent trip to India, and lets say.... er... things are different.

Indian cities are crowded and noisy; Traffic is insane and computers and broadband are still way beyond most indian homes. India is known as the IT hub of the world, and everybody on the street knows somebody who works in the IT industry. But wherever I went, there was one ubiquitous device making its presense felt :- the Mobile Phone. Almost everybody had one. I know people who dont have electricity in their homes, but yet carry a cellphone! At present India and China have the largest growth rates of cellular subscribers in the world. The availability of Prepaid or pay as you go services, where the subscriber does not have to commit to a long term contract, has helped fuel this growth on a monumental scale. Of course, as a percentage of total population, the numbers are nowhere near the developed countries.

India tops the world in online population growth, but only 20 million users are online, which is approximately 8 percent of the population.

Ring-tones, especially recent "bollywood" tunes were the craze, and the tendency of the recently "cell-phone rich" to show-off. I was travelling in the local train, a few people immediately wiped out their cellphones and started SMS-ing. One of my co-passengers received a phone-call, and the phone dutifully announced the event, with a very loud rendition of a hindi song. The recipient reluctantly answered the phone after waiting what seemed like an eternity to me! His intent was clear.

On the roads, travelling by car (driven by somebody else of course, i would be insane to drive there), with two-wheeler bikes zipping around, to my amazement, there was one bike-rider even talking on the cellphone while driving!! Incredible feats of recklessness. Mobile Phone company ads litter the side of the roadways. I saw a huge Yahoo! Go Billboard Ad inviting people to surf using their phones while stuck in traffic.

In India, over 80,000 persons die in the traffic crashes annually. While in the US, its roughly 44,000. The population of india is over 1 billion and of the US is roughly 300 million. So as a percentage of the population India has less traffic-related fatalities than the US, but travelling in india, you cannot but wonder how they manage to not kill themselves. That interesting statistics aside, back to cell-phones.

In India, SMS is the craze. Its free, unlike the US. From old aunts and uncles to young school kids, all of them know how to SMS. They get an incredible amount of talking done using the 11 or 12 keys available to press on a normal cell-phone. The user-interface of a phone is not ideal for such an interaction. Some analysts predict that voice-recognition will take over, instead of typing. But its not a very attractive solution for the young crowd, who use SMS as a diversion during class or as a tool for Romance. Some phone software allow for predictive word completion, but it can only do so much.

Using "Search" on phones is a different story. Piper Jaffray projects the global mobile search revenues will generate $11 billion by 2008. I think Voice Recognition can play a big role there. Even if the user-interface for data entry is improved and optimised, the form factor is still small, and so not very efficient. There are some start-ups that are moving in this direction, including PromptU and San-Diego based V-Enable. Google has already filed a patent for voice-based search - Voice interface for a Search Engine.

The Battle for Search is far from over, and there is room for a lot of improvement. It is an exciting future that i can foresee.

Create your own search engine or MyWeb?

2006-10-24T22:40:00.000-07:00

The date October 23rd, 2006. Google releases to the press their latest work. "Create your own search engine" the button says. The articles boast: "Dont have to have a PhD or a Masters to create your own search engine!". So i decided to check it out.

The site where its located is sort of vague :- http://google.com/coop/cse/ . After logging in using my google account, i had to find a name for my search engine, a description, submit tags to what the search engine was about, and submit sites that i wanted part of my search engine. The creation page also had a choice to search only those sites or to include the general web as well, and also allow other people to add sites to my search engine. To test it out, i added this blog site to the list of places to search.

The search didnt exactly work as i expected. I searched for "tufte", which i have obviously mentioned in my earlier blogs. It turned out zero results! That dissappointment aside, I think Google have set this up in a creative way. It allows users to customize their search engine page, allows it to be embedded in other webpages, and here is the best part : Allows you to make money (AdSense) if users click on Ads shown thru searching thru your search engine!!

But having used Yahoo!s MyWeb before, i couldnt help noticing certain similarities. Y! had released the original myweb in 2005 and in the summer of 2006 revamped the interface: http://myweb.yahoo.com . MyWeb also allows you to submit sites and tag them. If you are logged in to your yahoo account, a general search also searches your tagged sites, but only based on the tag. Contacts of yours can also see results and search the tags you have setup. With the y! toolbar, its a snap adding a new page to ur MyWeb.

The search battle continues. I have to agree that the way google is positioning their product, has gotten them a lot of press. Calling it "create your search engine" helps as well :-)

Whats in a name?

2006-09-09T23:03:00.000-07:00

Whats in a name? The famous cliched phrase. In my opinion, a lot!! While discussing the stock market over the regular afternoon caffeine shots with my colleagues, a particular conversation got me thinking. How do companies choose names?

So many Web sites and businesses have laid claim to English, Spanish and French terms that few words are left. Thats why some companies have resorted to making up their own names. Accenture, Agilent, Altria and Verizon are just a few recent examples.

Companies spent millions trying to come up the right name. There are consultancies that help other companies with names. Terry Heckler, founder of Seattle-based Heckler Associates, which helped name Starbucks, Cinnabon, Loudeye and Panera Bread, said companies have limited options these days. Another company called NameBase does naming consultancy, and have big-names on the payroll.

Creating names with unique sounds is something of a science. Memorability, alliteration and vowel harmony are extremely important. Picking a name that grows along with the business and effectively describes what services the company provides still remains a challenge. Here is a nice how-to I found on the web.

I took the effort to check the names of companies listed on the NASDAQ and the first letter of the name. It is an interesting statistic. As of today, here is the distribution for the starting letter frequency :-

Over 10% start with the letter C, and an almost equal percentage start with A. There seems to be an overwhelming predilection to use names starting with A or C.

Meanwhile, the letter frequency of the most common first letters in an english word is of the following order (highest to lowest) :-

t o a w b c d s f m r h i y e g l n o u j k

The letter Frequency of the most common last letter in a word is of the following order :

e s t d n r y f l o g h a k m p u w

(Interested in trivia like that? This article may be to your liking)

There are even identifiable patterns in what experts think comprises a good-sounding company name. One blogger refers it as the ASYPTA pattern. According to this pattern, the name is of this format :- A, consonant, vowel, consonant, A (perfect example :- Ariba). Wonder what MicroSoft was thinking? Who wants a company associated with micro & soft?? To its credit, the company is now beyond its name. It has become the antonym of what its name suggests.

Please suggest company names that you think are good in the comments.

User Interface Languages

2006-09-08T00:46:00.000-07:00

It is an undisputed fact that user-interfaces for software took a quantum leap with the Apple Macintosh computer. Before that we were squinting at green phosphorous elements as each character appeared on the CRT. Ambitious developers tried to make the best use of it by using various ASCII characters to represent boxes and buttons. As user interfaces improved, and most applications started having buttons and windows and forms (except grocery counter software, and maybe bank applications), developers started finding themselves using similar constructs to specify UI elements. Most developers now understand the meaning of terms like dialog boxes, buttons, lists, checkbox, editboxes and the like. For a nice history of the GUI, read this article - Of Mice & Menus.

If one looks at the development of computer languages, we can see a similar trend towards abstraction. Assembly language gave way to higher level languages like C++, which allowed developers to abstract commonalities to build more complex applications. As UI-based Apps became more complex, developers were desperately looking for ways to decouple presentation code from logic. It started with the MVC Architecture, mainly in Desktop Apps. In the case of UI, the common elements were standardized and abstracted, and languages were developed to specify those UI elements. With the rise of internet, Internet-based UI apps are starting to take off. CSS or Cascading Style Sheets is an excellent step in that direction. Rich Internet Applications or RIA, as they are known, came to life with the ubiquitous XMLHTTPRequest object, google maps and the birth of AJAX.

Standardization is everywhere. Look at IM (Instant Messenger) conversations: brb, lol, asap, afaik, ttyl have all been sort of standardized. Its no wonder that developers tried to standardize UI elements, and developed languages to abstract it. Though it started with a few, now there is a barrage of language specs claiming to describe User-Interface Elements. One such language is XUL.

XUL or XML User-interface Language is sponsored by mozilla. It uses XML syntax to describe UI elements, and allows embedded scripts to define behaviours. Microsoft is trying something similar with XAML. They have grander schemes of using it as an application markup language.

Here is a brief list of user interface languages. One of them particularly sparked my interest... Laszlo. It is a weird name. Originally developed by a privately held company called Laszlo Systems, headquartered in San Mateo, California. It was made into an Open Source Development recently at http://www.openlaszlo.org. IBM also supported it and were crucial in the initial development of IDELaszlo, an eclipse plugin. The product requires a Laszlos server, which compiles the laszlo script into flash or limited support of DHTML, and SVG coming soon.

Speaking of SVG....wonder whats holding it back? Flash seems to be so popular, but then its just feeding money into Adobe, and their plan for world domination. Its even supported by the W3C, but so far only FF and maybe Opera, support it natively for now. SVG is a technology i look forward to, once it becomes a standard on all browsers.

Visualization of Information

2006-08-10T23:20:00.000-07:00

I recently felt that a mouse is getting obsolete. It is not that the mouse itself has not seen any innovation. It has. The dirty and inaccurate ball has been replaced with a laser light, trackballs, graphic tablets. But it still creates carpel tunnel syndrome and fatigues the arm. We still find it hard to move that pointer to the exact "X" on the window ,and if by chance we missed the "X", something bad happens. It is embarassing that the most advanced species on earth needs to communicate with the most amazing inventions on earth through a small rodent! A new interface needs to be developed.

Anyways, one thing lead to another, and i started reading "Envisioning Information" from Edward R. Tufte. An interesting book, its about how humans have captured various data and tried to share that data with others by representing in various forms on paper.

Tufte's books include The Visual Display of Quantitative Information, Envisioning Information, and Visual Explanations: Images and Quantities, Evidence and Narrative.

The problem with presenting information, as explained by Tufte, is simple -- the world is high-dimensional, but our displays are not. To address this basic problem, Tufte offered five principles of information design:

Quantitative thinking comes down to one question: Compared to what?
Try very hard to show cause and effect.
Don't break up evidence by accidents of means of production.
The world is multivariant, so the display should be high-dimensional.
The presentation stands and falls on the quality, relevance, and integrity of the content.

The book also shows how Galileo mapped sunspots on a circular disk and recorded the same over many years, which helped him see a trend. It shows how the japanese used train maps on paper to show multi-variate data. Take a look at how the train map was created:

How many variables are captured in that diagram? I am sure we could be more clever and add more dimensions to it. But it can quickly be overwhelming.

People have come up with ingenious methods of solving the problem of showing multi-dimensional data on two-dimensional screen/paper (Flatlands).

Here is an interesting article about a talk given by Tufte.

The latest book by Tufte is called "Beautiful Evidence".

More later.

Popularity of my blog overtime :-

Beginning of the Beginning

2006-08-10T23:04:00.000-07:00

First Post