b(ond)log

Is it cool to be on twitter?

I end up reading about facebook and Twitter more than I need to for social or work reasons. However, it will surprising to know that is more for intellectual curiosity for the underlying technologies and the infrastructure and scalability issues than for the social networking.

I was once bitten by the Ruby on Rails bug and developed a couple of public apps and a few internal applications using the framework. I ended up liking the Ruby language while I was doing it. I am a fan of Rapid Application Development tools (especially for web sites with database driven backeds) that still feel like basic programming. Java with its sea of technologies and packages never appealed to me. I remember reading an early Ruby slide that said if the language and compiler are not fast enough just wait for hardware to get faster. (Music to hardware designers ears).

So I was doing a lot of rubbernecking when twitter was having problems. I am still waiting to try out Scala after I saw the raging debates on the internet about languages and speed that I had not seen since the C, Java and Perl days and read about Twitter and Scala.

When you are developing rails apps and you are hacking your way around, it is fine to get a bunch of errors and screen dump of what went wrong. However, when you start deploying it and want to convince users to accept the new system exceptions are like segfaults. You do not want them to see gobs of exception messages on their browser which will be an excuse for them to write it off even if they did not RTM and did something wrong. Another problem is that people will happily hit Back and not save what happened and only bitch about the instability of the system.

Exception Notifier is a great plugin to solve these issues. Basic installation of the plugin is pretty smooth and if you already have emails working from your app then exception notification works pretty easily. There are enough Google results for you to get it working.

One standard problem that almost everyone has is to get it working in your development environment. The following changes did it for me: update your application.rb and update your config/environment/development.rb.

application.rb

include ExceptionNotifiable
local_addresses.clear


development.rb

config.action_controller.consider_all_requests_local = false


More interesting changes are to do with picking specific errors out of all the errors and sending some to the browser and some via email. In my case, I had support for free-form SQL queries in our internal application which could give internal MySQL errors to user queries which I wanted to go to the browser. Here is what you add to application.rb:


# Try to get incorrect SQL errors and show them on screen
def rescue_action_in_public(exception)
case exception
when ActiveRecord::StatementInvalid
render :text => "You have errors in your SQL Syntax. Please try the Back button and fix them before trying again. #{exception.message.inspect}"
when ActiveRecord::MultiparameterAssignmentErrors
render :text => "You seem to have some inconsistent input. e.g. Date like Feb 30 2007. Please try the Back button and fix them before trying again. #{exception.message.inspect}"
else
super
end
end


The ActiveRecord::MultiparameterAssignmentErrors is a very interesting exception which you surely want to get to the user than via an email to the admin. I found out that when forms have incorrect date input (like Feb 31), MySQL generates this obscure exception. I could have borrowed some date validation code but why do extra work when you can rap the users knuckles instead.

When we construct views of query results using ERb in rails, most of the time we know what we are displaying and how to format it. However, there are a few occasions when we run find_by_sql queries. Even rare are the occasions when we let users type free-form SQL queries into an input box and just take the query string as is and run it on the database (MySQL in my case).

The reason I did the free-form queries in our internal site was to let users crunch the data the way they wanted instead of trying to add more and more views as they kept requesting them. However, this presents a problem in the View portion (of the Model-View-Controller).

Since we do not know what columns the user has actually requested in the query, it is not easy to figure out how to write the ERb/HTML view code. Fortunately, the attribute_names method of the model helps here.

So you do this in your controller:

def sqlquery
  if (!querystr.blank?)
    @results = Model.find_by_sql(querystr)
    @total = @results.size()
     if (@total > 0)
       @columns = @results.first.attribute_names
     end
   else
      @total = 0
   end
end

The attribute_names is very useful since it returns exactly the columns that are requested in the SELECT portion of the query (rather than every column in the Model model). Also, when the SELECT uses a AS clause (please refer to the MySQL SQL documentation), the attribute_names contains the string in the AS clause. e.g. If the query has SELECT name AS ‘UserName’ … then attribute_names returns UserName as one of the columns.

However, I still faced one gotcha that I could not figure out either by searching or by reading the rails source. The attribute_names method does not return columns in the order they were specified in the SELECT clause. It can return them sorted alphabetically, but that’s it! Now, users do not like see columns in random order showing up in the results table. I thought of parsing the query but then realized it was futile since the query could contain recursive SQL and table joins etc. The easiest hack that I found is to just try string matching which surprisingly works for all our SQL queries. I do not see a reason why it should not for all cases. So the new code looks like this:

def sqlquery
  if (!querystr.blank?)
    @results = Model.find_by_sql(querystr)
    @total = @results.size()
    if (@total > 0)
        cols = @results.first.attribute_names
        if (querystring.match(/\s*SELECT\s*\*\s*FROM.*/i))
            @columns = cols
        else
            pos = cols.inject({}) { |h, col| h[col] = querystr.index(col); h}
            @columns = cols.sort { |x,y| pos[x] < => pos[y]}
        end
     else
         @total = 0
     end
end

So what does the code do? First, we check if the query was a SELECT * FROM which will not have any column names in the SELECT clause. Then we just rely on the random order of the columns returned by attribute_names. If not, we compute the position of the strings returned by the attribute_names in the original query (in the cols.inject line). They could be simple column names or even the AS clauses. We can now use the position(index) of the substring as a sorting criteria so that the earlier occurring column names in the SELECT clause is earlier in the @columns array. This seems like a hack but it seems to work everytime. If there is a more elegant way, I am all ears…

Now in the view, we can use the @columns array for our view (do not forget to change the code below from table to <table>, td to <td>, etc.):

table
thead
  < % for col in @columns %>
       td < %= col %> /td
   < % end %>
   /tr
/thead
      < % for item in @results %>
        tr
        < % for col in @columns %>
          td < %= item[col] %> /td
        < % end %>
        /tr
      < % end %>
/table

I have been dabbling with Ruby on Rails a little more lately. I have hacked around and figured out solutions for issues that stumped me based on posts from others or from experiments. I will try to catalog some of those in my blog so it is useful for others. Pardon me if some of these seem trivial or violate Ruby idioms. I have been learning Ruby as I wrote an internal application in RoR. Recently, I ended up coding and checking in stuff in Verilog, C++, Ruby and Javascript in a single day so I have valid excuses to muck up all of them.

The first one is a hack I figured out to display find_by_sql results that I could not even get answers from rails mailing list. Stay tuned…

Even though I work in a chip company, I believe in tools that work on the web so creating, updating and maintaining data is a collaborative process (and less personal overhead ). Having a spouse who works in web tech is of extra help. I was one of the early adopters of the Twiki at my company (Andy was the initiator). One of the earliest tools that was born out of my pain of having to create work plans, track them and collect status info was the Project Planner Plugin for Twiki (which I also released to the public domain).

I have done a few other CGI scripts to interface databases to the web along the way. But, cgi scripting in perl is not my idea of writing software. It keeps getting in the way more often than letting me do what I want to. Of course, I am not a master of the language. People actually commented on the Project Planner plugin that the code was easy to understand since it was perl written in C style!

So I stumbled upon Ruby on Rails accidentally and that led to making the softwareinterview.com website. I really did it like the 15 minute video on the RoR website and it works! We spent more time on the template and CSS for the view. Yeah, and it is true that getting the first few things up is as quick as 15 minutes but you really do need to understand framework, Ruby, ERb and a few other things to be really up and running. Pickaxe and DHH to the rescue. It helps to read and do some development as opposed to getting dazed with just hacking away and getting stuck on how to take it to the next level.

It also led to me adopting Rails for a cgi script I was maintaining at work. It was a small perl cgi script and was moderately well architected (YAML for config files, serializing and deserializing objects, etc.). However, it was getting a pain to maintain and add any features. I did keep finding myself repeating the same code, trying to abstract it away, refactor it, etc.

I re-implemented the whole thing in RoR and it did make development faster. Since I was not a perl-cgi wiz, getting stuck at RoR and perl-cgi was pretty much similar overhead in trying to search for an answer. It was easier in RoR once I found the answer since it was so much more succint in RoR as opposed to perl-cgi. The application though small does use many features:

• multiple tables
• foreign keys
• ActionMailer
• acts_as_versioned
• observe_form AJAX for searches

And as you can guess I did run into several itty-bitty problems throughout (especially since I was a Ruby Nooby too). Surprisingly, web search seems to provide many answers since there are many new rails pages and not too much noise. I find that searching for many topics on the web finds more noise than answers.

About that list of small issues:

• collections in forms
• AJAX for form vs fields
• choicebox selection list and selected value

Oh! As a side effect to the whole thing, I love Ruby! A fortune sample reads, “Computers do what you tell them to do, not what you want them to do.” The way I see it, I have always had a language that let me tell the computer what to do – C. It is pretty close to really thinking in terms of a processor chip.

add 1 to i, mov i to memory at location m

I have never really found a language that really lets me write software that I want to do. Ruby seems to come pretty close to writing code the way we write pseudo-code when writing algorithms.

for each of the elements in the array, slice ‘em, dice ‘em and cook ‘em.

Really! That is how I feel writing in Ruby. I am neither a master of languages nor a novice. I have written and graded a few programs in LISP, COBOL, Perl, Prolog, etc. to understand concepts behind computer languages (PCP did teach us the wonderful lambda calculus in undergrad). Perl seems like sed+awk and I haven’t written anything in python but I do not like white spaces dancing around me in my nightmares. I had enough of those with COBOL (spaces and nightmares while sleeping in the undergrad computer lab).

I design processors (albeit writing a lot of software) and not web sites. Even if I do not use RoR a lot, at the least RoR gets credit for my stumbling onto Ruby! When I have a choice, I will be scripting mostly in Ruby.