Aawaara

Lucknow Elections - The Good, the Bad and the Ugly

Until now it has mostly been about information dissemination and preaching, but as of now it’s about making a decision. And when it comes to making a decision, I want to make an informed decision (at least I want to think so), post evaluating the pros and cons. Lucknow goes to vote today, and the candidates in fray are Rajnath Singh from BJP, Javed Jaffrey from AAP and Abhishek Mishra from SP.

As I think of Rajnath Singh, the BJP president, the earliest bad memory that strikes me, is of him being in tears when Raghuraj Pratap Singh alias Raja Bhaiyya, the dreaded mafia don from Kunda was arrested by the Mayavati government in 2002. While the common man was happy in the sub-conscious (we were not brave enough to be vocal then, and social networks were still in their infancy), and praising the guts of Mayavati, this guy was crying out loud in a press conference, attributing this as an attack on “Thakurs” which he himself is proud to be, with total disregard to credentials/doings of Raja Bhaiyya. Once he was out of jail, the police officer who led the raid on his house died in an accident, and till this date, no one has gone to jail for that:

http://www.firstpost.com/politics/sps-murder-why-we-can-never-underestimate-raja-bhaiya-647797.html

Though this is a reason enough for me to not vote for him, given his petty caste based politics, but I will reconsider given the thousands of crores of advertising money that is being flushed down my throat, which says - “Achhey din aaney waaley hain!”. On second thought, this guy has run away from Ghaziabad, just to mitigate the risk of him losing from there. I have checked with friends living in Ghaziabad, and they confirmed that he did nothing (even ardent BJP fans living in Ghazibad confirmed this). And if we go purely by statistics, he has performed poorly:

http://ibnlive.in.com/news/mps-rating-rajnaths-performance-below-average/458776-3.html

With Rajnath Singh out, next is Abhishek Misra from SP. Though SP is not worth considering, but this guy certainly has the credentials - he is a PHD from University of Cambridge, and an ex IIM-A professor. But the sad news is that he is the sitting MLA from my constituency, and he hasn’t done anything great (yet). Additionally, just before the assembly elections I had emailed him asking why should I vote for him. He never wrote back. Seemingly, he doesn’t give a f#@k about my vote, so why should I! Negated!

Next is Javed Jaffrey from AAP. The primary argument against him is that he is a comedian. I am a software engineer. Someone else is a Banking professional. Is there something wrong with being a professional. And I bet he is a good comedian, and is good at what he does. I have laughed numerous times listening to him. Being good at his profession is a plus for me. And people from all walks of life entering politics is good as well. I personally believe that until you, me and the person living next door isn’t going to enter politics, nothing is going to change. Until now politics had been the last resort of people who couldn’t do anything with their life (minus rare exceptions in the sea of incompetent political leaders and the so called cadres). I am glad that this changed with AAP, and the common man dared to talk and walk politics. For many now politics is a matter of choice, not just a way to hoard money and power, just because they weren’t good at anything else. They left what they were good at, just for greater good. Don’t let them reconsider their decision, thinking this was in vain, and they are not fit for this (perceived) shithole. Though there can be bad fish in them, but based on that reasoning alone, I am not going to throw the baby out with the bath water. I am an optimist, and I hope for a better India soon enough.

Additionally, I found this google hangout video of Javed Jaffery convincing enough of his intentions:

https://www.youtube.com/watch?v=DHnACIRL3Zo

If he doesn’t do anything for his constituency just like the Govinda’s of our age, I will be the first one to shout out loud. Any other candidate who I haven’t mentioned here, is either not worth mentioning, or wasting words on, or isn’t popular enough.

Until now, I have always cribbed about not having a good option to vote for. All political discussions started at “sab chor hain”, and even finished at that. This time around, I have an option, and I am going to make full use of that opportunity. If you are going to vote the same way you did in the past, the way your parents told you to, and their parents before that, don’t crib later ki “iss desh ka kuchh nahi ho sakta”, primarily because “tum nahi chaahtey ki iss desh ka kuchh ho”. Vote wisely, and decide for yourself (it’s time you grew up, and grew out of caste, creed and religion). Some people will try to persuade you that you are wasting your vote on a losing candidate, but do remember that if enough people waste their vote on the losing candidate, then that candidate wins!

PS: Excuse the typos, as I am typing this in a train, and the lights are off. All the best Lucknow! Excited to vote tomorrow!!!

That slap did hurt!

I find it sad (and amusing at the same time) when people make fun of Arvind Kejriwal being attacked or slapped (irrespective of whom you will vote for), and call it a self orchestrated event. For a while, imagine yourself being slapped - and the sudden splurge of adrenaline that you will experience, and then rephrase your thinking. I know you wont, until and unless you get slapped by a politician, and re-evaluate the powerlessness of an AAM Aadmi, and the toothlessness of a system, which suddenly you have started believing in, and have utmost respect for! These days nothing is better than Indian police, law, judiciary and criminal politicians. At least it feels so from your reactions - whereas in the past, it started at “sab chor hain”…..and ended at that as well.

The good thing (for me) is that this AAM Aadmi is undeterred, and at least doing something to change the current state of affairs, whether you like it or not! Meanwhile, hopefully, you wouldn’t get slapped some day by someone powerful - who tells you “tuu jaanta nahi hai ki mai kaun hoon”!



Today Arvind Kejriwal wasn’t slapped. It was you and me who was slapped again, and I can feel the pinch. May be you cannot! As a mark of respect to this great soul, and his undying fighting spirit, and just for the sake of making a statement - I am making this non anonymous contribution to the cause. #iStandWithArvind

May be no birthday treats this year, but this is much more important than you or me! Critics and nation builders, whose sole responsibility is to save this world from the so said AAPtards, mock me when this is over, as you want it to be. Laugh at me when your “ram rajya” is established (and nothing changes, then do repeat iss desh ka kuchh nahi ho sakta), but I am happy and content that at least someone challenged the status quo, stood up, and at least tried. It’s easier to talk, than do. It’s easier to take, than give. At the end of the day, you will get what you deserve, and it will be decided the way you vote!

As for the थप्पड़ शूरवीरs (from a post in FB):

“है हिम्मत तो घूस माँगने वाले पुलिस वाले को थप्पड़ मारो,
दफ्तर में फ़ाइल उठाने की कीमत मांगने वाले चपरासी को थप्पड़ मारो,
अपने ऑटोग्राफ देने की कीमत माँगने वाले अधिकारी को थप्पड़ मारो,

हो हिम्मत तो 500 रुपये में तुम्हारा वोट खरीदने वाले को थप्पड़ मारो,
सड़क में, बस में मेट्रो में, बहनो को परेशान करने वाले लडको को थप्पड़ मारो,

हर बात में धमकी देने वाले, नेता के चमचे को थप्पड़ मारो, और घोटाले करने वाले नेता को थप्पड़ मारो,
पर नहीं,
गुलाम हो तुम, डरपोक हो तुम।
तुम सिर्फ आम आदमी को मार सकते हो, अरविन्द को मार सकते हो”

Smallest piece of code that’s going to change the world (okay iOS world may be) :P

The iOS custom URL scheme is broken. Have a link to a custom url for your app in the browser, and it works like a charm, if you have the app installed. If not, then you are doomed! Safari shows a super helpful alert that says - ‘Safari cannot open the page because the address is invalid.’ The other browsers are even more helpful, and they do nothing! Absolutely nothing. They don’t have the decency to redirect you to an alternate URL (or provide one), or prompt you to download the app.

May be the solution is somewhere out there, but I couldn’t find it. Or may be google has been lazy lately, and their focus shifted to google+ =P

There are start-ups built around solving this problem, and I am going to give this away for free. Remember, I wanted to change the world ;)

To start with, this isn’t rocket science, but it took me a lot of trial and error, and fine tuning to get this working on all the iOS browsers. In safari, it still shows that alert, but it goes away in a couple of seconds when the alternate URL is loaded. Here is how I went about solving this:

If it ended here, then life would have been so much fun. But it’s like saying the world would have been so much better if there was no internet explorer. Simply put, it isn’t!

The bad news is that the above code works only for safari, because they designed iOS and came up with 'pageshow’ and 'pagehide’ events, so they have them working well. The other browsers don’t care or give a shit. They don’t fire those events on tab change, or when the browser switches to background/foreground: http://stackoverflow.com/questions/21311478/pagehide-and-pageshow-events-dont-work-as-expected-on-ios-chrome

Sad, but true! Not a problem. Below is another javascript snippet that works well on all the browsers (I tested safari, chrome and mercury, but logically it should work on others as well).

I increased the timeout value to 2000ms because chrome sometimes takes a bit too long to launch the app (may be tries to do some post processing magic), and triggers the code in timeout block before the timers are cleared. Tweak it as per your need, or you can even have timeouts per browser. Also, don’t forget to put the 'source_url’ class on your link tag.

That’s it. This should work in android as well (may be there are other events available in android), but I’ll cross that bridge when I am there. You have the tools now.

Let me know if you need an explanation, see some failure scenarios, or if I can make this better, or you just want to say thank you. If this really changed your world, as mentioned in the misleading title, then send me money instead =P

How to lose $172,222 a second for 45 minutes

That’s why ;)

pythonsweetness:

This is probably the most painful bug report I’ve ever read, describing in glorious technicolor the steps leading to Knight Capital’s $465m trading loss due to a software bug that struck late last year, effectively bankrupting the company.

The tale has all the hallmarks of technical debt in a huge, unmaintained, bitrotten codebase (the bug itself due to code that hadn’t been used for 8 years), and a really poor, undisciplined devops story.

Highlights:

To enable its customers’ participation in the Retail Liquidity Program (“RLP”) at the New York Stock Exchange,5 which was scheduled to commence on August 1, 2012, Knight made a number of changes to its systems and software code related to its order handling processes. These changes included developing and deploying new software code in SMARS. SMARS is an automated, high speed, algorithmic router that sends orders into the market for execution. A core function of SMARS is to receive orders passed from other components of Knight’s trading platform (“parent” orders) and then, as needed based on the available liquidity, send one or more representative (or “child”) orders to external venues for execution.

13. Upon deployment, the new RLP code in SMARS was intended to replace unused code in the relevant portion of the order router. This unused code previously had been used for functionality called “Power Peg,” which Knight had discontinued using many years earlier. Despite the lack of use, the Power Peg functionality remained present and callable at the time of the RLP deployment. The new RLP code also repurposed a flag that was formerly used to activate the Power Peg code. Knight intended to delete the Power Peg code so that when this flag was set to “yes,” the new RLP functionality—rather than Power Peg—would be engaged.

14. When Knight used the Power Peg code previously, as child orders were executed, a cumulative quantity function counted the number of shares of the parent order that had been executed. This feature instructed the code to stop routing child orders after the parent order had been filled completely. In 2003, Knight ceased using the Power Peg functionality. In 2005, Knight moved the tracking of cumulative shares function in the Power Peg code to an earlier point in the SMARS code sequence. Knight did not retest the Power Peg code after moving the cumulative quantity function to determine whether Power Peg would still function correctly if called.

15. Beginning on July 27, 2012, Knight deployed the new RLP code in SMARS in stages by placing it on a limited number of servers in SMARS on successive days. During the deployment of the new code, however, one of Knight’s technicians did not copy the new code to one of the eight SMARS computer servers. Knight did not have a second technician review this deployment and no one at Knight realized that the Power Peg code had not been removed from the eighth server, nor the new RLP code added. Knight had no written procedures that required such a review.

16. On August 1, Knight received orders from broker-dealers whose customers were eligible to participate in the RLP. The seven servers that received the new code processed these orders correctly. However, orders sent with the repurposed flag to the eighth server triggered the defective Power Peg code still present on that server. As a result, this server began sending child orders to certain trading centers for execution.

19. On August 1, Knight also received orders eligible for the RLP but that were designated for pre-market trading.6 SMARS processed these orders and, beginning at approximately 8:01 a.m. ET, an internal system at Knight generated automated e-mail messages (called “BNET rejects”) that referenced SMARS and identified an error described as “Power Peg disabled.” Knight’s system sent 97 of these e-mail messages to a group of Knight personnel before the 9:30 a.m. market open. Knight did not design these types of messages to be system alerts, and Knight personnel generally did not review them when they were received

It gets better:

27. On August 1, Knight did not have supervisory procedures concerning incident response. More specifically, Knight did not have supervisory procedures to guide its relevant personnel when significant issues developed. On August 1, Knight relied primarily on its technology team to attempt to identify and address the SMARS problem in a live trading environment. Knight’s system continued to send millions of child orders while its personnel attempted to identify the source of the problem. In one of its attempts to address the problem, Knight uninstalled the new RLP code from the seven servers where it had been deployed correctly. This action worsened the problem, causing additional incoming parent orders to activate the Power Peg code that was present on those servers, similar to what had already occurred on the eighth server.

The remainder of the document is definitely worth a read, but importantly recommends new human processes to avoid a similar tragedy. None of the ops failures leading to the bug were related to humans, but rather, due to most likely horrible deployment scripts and woeful production monitoring. What kind of cowboy shop doesn’t even have monitoring to ensure a cluster is running a consistent software release!? Not to mention deployment scripts that check return codes..

We can also only hope that references to “written test procedures” for the unused code refer to systematic tests, as opposed to a 10 year old wiki page.

The best part is the fine: $12m, despite the resulting audit also revealing that the system was systematically sending naked shorts.

[Edit: seems this was posted a little too quickly: the final loss was $460m and the code was dead for closer to 9 years, not 8]

How to get created_at timestamp from mongo BSON id using mongoid

Writing this down as I always forget this, and this search query is not very google friendly either (and I usually waste a lot of time looking for this, and probably so would others). This is how you do it:

sample_object.id.generation_time

Now the bonus stuff! If you wan to do range queries for created_at timestamp on the BSON id, this is how you do it:

Break it down!

Wiser people have said - if you have a problem, break it down! If the problem is bigger, break it down further. This is true both philosophically and technologically.

I’ll describe a technological incident here in the hope that it could help someone else as well (when they land in this tricky situation, and are looking for a quick resolution). We use delayed_job gem in our web apps for asynchronous processing. It has performed pretty well for us all this while, but one fine day it caved in. We had a barrage of jobs into the dealyed_job table, and the job processing daemons were just unbearably slow. The reason being that there were 600K jobs in the queue, and the rate at which dealyed_job daemons fire the mysql queries every 5 seconds was too much I/O for the database server. Soon the slow query logs were flooded with update queries. I tried building a few indexes to speed up some of those queries, but the improvements weren’t significant enough. What now? I filed a bug on the github repo in hope of greater good for future users, but that wasn’t sufficient enough. Those jobs in the queue were time sensitive in nature, and at the current speed would take days, if not weeks to clear out. We needed a quick fix now, and a long term fix (better solution as compared to delayed_job) later.

Solution (quick fix): I went back in time and realized that delayed_job had performed well for a few thousand jobs in the table. That’s it - we needed to pause, and break it down a bit - in this case, the mysql table. So we copied all the data from delayed_jobs table to a new table (backlog_jobs) and cleared the original table (the delayed_job table from where the jobs are picked). Then we copied the first 5000 jobs from backlog_jobs to delayed_jobs and deleted them from backlog_jobs. They were processed pretty quickly in a matter of minutes. Wallah! This was it. Then we wrote a small cron than runs every 1 minute and checks the count of delayed_jobs table. If there are less than 100 rows, it repeats the process, i.e. copies the next batch of 5000 jobs, and deletes it from backup_jobs. This way I got back to the older performance levels. Below are some handy queries that you will find useful:

# To create a copy of table
CREATE table backlog_jobs like delayed_jobs;
INSERT into backlog_jobs (SELECT * from delayed_jobs);
DELETE from delayed_jobs;

# To copy jobs in batches
INSERT into delayed_jobs (SELECT * from backlog_jobs order by id asc limit 5000);
DELETE from backlog_jobs order by id asc limit 5000;

So the next time you face database related issues, break it down a bit. That’s what database partitioning is all about, and that is what everything finally comes down to ;)

Would like to hear what I may have done wrong to start with, and what I could have done better. Don’t hesitate to teach me a lesson (pun intended)… :P Philosophically speaking, life is an eternal learning process ;)

Why linode rocks!

I’ll keep it short and sweet:

which translates to no downtime for the past 898 days (and I didn’t need to restart my server for ~ 2 years, 5 months). Isn’t that f#$@ing amazing!

But sadly this streak ends today, and I am restarting the server to avail some upgrades - @2x RAM, @2x processor cores and @10x data transfer (didn’t need a restart to get this one) and BillBaba.com will be down for the next few minutes.

Price - FREE

That precisely sums it up why Linode is awesome - excellent service, free upgrades and amazing support!

I would recommend it any time of the day. If you are planning to set up a server, click here (shameless plug - and help me earn some free credits).

Rails 3: Custom validation messages for the same column name using some other column value

Problem: We wanted to add custom validation message, and call the same mysql table column with different names depending on the value of some other field. After much googling I couldn’t find any suitable answers. Posting the solution I came up with, so that it can be of help to others:

Solution: 

I couldn’t find solutions using procs, interpolations, etc. I wrote a custom validator, as all the columns are available there. Below is the sample code:

Please note that linktype and url are columns in the links table.

Also answered a related question on Stackoverflow here:

http://stackoverflow.com/questions/5085284/how-can-i-interpolate-ivars-into-rails-i18n-strings-during-validation/13881719#13881719

And suddenly I lost all the gems, that I had collected over time!

This isn’t about the material riches obtained over time, but about the shared wealth among the Ruby community ;)

In case you notice that suddenly all your gems vanish from your system, and “gem list” shows none of the gems that you had installed previously, the details below can help you. In retrospect, I remembered that this was a result of gem update —system,that I had executed some time back. It updated rubygems version, as well as the ruby version used by rubygems, and it was a clean slate (I don’t know why they mention that command before installing a gem, in documentation. It can ruin your day!).

So effectively, I had two executables - gem1.8 and gem1.9.1 in my bin directory. The update had made a soft link: /usr/bin/gem -> /usr/bin/gem1.9.1. You can check which executable is being used by doing which gem, and then ls -al <gem path> to get the executable.

$ gem env

RubyGems Environment:
  - RUBYGEMS VERSION: 1.3.7
  - RUBY VERSION: 1.9.2 (2011-07-09 patchlevel 290) [i686-linux]
  - INSTALLATION DIRECTORY: /var/lib/gems/1.9.1
  - RUBY EXECUTABLE: /usr/bin/ruby1.9.1
  - EXECUTABLE DIRECTORY: /usr/local/bin
  - RUBYGEMS PLATFORMS:
    - ruby
    - x86-linux
  - GEM PATHS:
     - /var/lib/gems/1.9.1
     - /home/amit/.gem/ruby/1.9.1
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :benchmark => false
     - :backtrace => false
     - :bulk_threshold => 1000
  - REMOTE SOURCES:
     - http://rubygems.org/

$ gem1.8 env

RubyGems Environment:
  - RUBYGEMS VERSION: 1.8.15
  - RUBY VERSION: 1.8.7 (2011-06-30 patchlevel 352) [i686-linux]
  - INSTALLATION DIRECTORY: /usr/lib/ruby/gems/1.8
  - RUBY EXECUTABLE: /usr/bin/ruby1.8
  - EXECUTABLE DIRECTORY: /usr/bin
  - RUBYGEMS PLATFORMS:
    - ruby
    - x86-linux
  - GEM PATHS:
     - /usr/lib/ruby/gems/1.8
     - /home/amit/.gem/ruby/1.8
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :benchmark => false
     - :backtrace => false
     - :bulk_threshold => 1000
  - REMOTE SOURCES:
     - http://rubygems.org/

The Fix:

Create a soft link for the gem1.8 executable, and everything will be as it was:

rm usr/bin/gem

ln -s /usr/bin/gem1.8  /usr/bin/gem

Back to the good old days! I’ll save the rubygems update for some other day, and I dread gem update —system form now on…. :O :O :O

BillBaba: And a year flew by!

On a usual Monday afternoon the phone beeped, and I realized that BillBaba.com had completed one year since launch. The initial yearly reminder that I had set up to remind me of this had actually played it’s part and it dawned upon me that a year had passed since BillBaba.com said Hello to the world wide web!

I would lie if I said that the year passed in a flash. The year was longer than all those years when going to office was a routine, it was heavier, but it was faster. The Einstein’s theory of relativity does apply to Entrepreneurship as well - at faster speeds (comparable to speed of light), time does appear to slow down! Overall it has been a great learning experience, and definitely has carved out a better me.

This post if more about data, about what BillBaba.com has seen in terms of numbers. Personally, I have always felt that there is a lack of data points in the India eco-system. We don’t know about many successful exits, and even if we have heard of a few, we absolutely don’t know about the financial value of the transaction. There aren’t enough benchmarks to see if you have good, decent or poor traffic/userbase, especially in the Indian context. Below is my contribution to the eco-system.

All data as on 25th June, 2012 (1 year since launch):

Total users: 3,113

Total reminder SMS sent: 43,518

Total reminder email sent: 62,239Total Facebook reminders sent: 27,008

Unique Visitors: 13,252 + 952 (blog + forum)

Visits: 18,346 + 1079 (blog + forum)

Page Views: 92,663 + 2025 (blog + forum)

Thanks to everyone who has stood by me :)