Leonard Teo » Web Development

2012 Resolution: Retire Internet Explorer 8

Leo — Sat, 31 Dec 2011 14:45:51 +0000

One of the biggest frustrations that I have as a professional web developer is Internet Explorer – more specifically IE8, which is currently refusing to die because it is “good enough but not quite”. Here we are, trying to move forward with new technology on HTML5 standards that work across most modern browsers, except things don’t look or work quite right in IE8. Corporations still give out computers to employees with IE8 pre-installed. Enough is enough. Let’s make 2012 the year that we, as a body of web professionals, begin to retire IE8.

How?

From now on, every site that I deliver for clients will have, by default, an IE8nomore header (or variant), which will only show up when a user with IE8 or less browses to the site. If a client specifically wants me to support IE8, they will need to pay extra for it. It’s that simple. Clients need to be educated that IE8 is just bad software. If they have the chance to tell you that a site you created doesn’t work in IE8, you’re too late. You need to put the header in by default so that when they, or their colleagues browse to your site, they immediately see that their browser is outdated. It’s psychological. They need to upgrade. You don’t need to lower your standards.

If you’re a professional web developer, make it your resolution. Let’s rid the world of standard-less browsers. Let’s nail IE8 and please be done with it.

Links

IE8nomore header
IE6nomore header (the original)

Rails, Bootstrap and will_paginate

Leo — Thu, 29 Dec 2011 14:13:55 +0000

If you’re using Ruby on Rails, you’ll likely use the will_paginate plugin. If you use Twitter’s Bootstrap CSS framework as well, you’ll notice that the pagination is all messed up. This gist fixes it:

  # application_helper.rb
  # https://gist.github.com/1205828
  # Based on https://gist.github.com/1182136
  class BootstrapLinkRenderer < ::WillPaginate::ViewHelpers::LinkRenderer
    protected
 
    def html_container(html)
      tag :div, tag(:ul, html), container_attributes
    end
 
    def page_number(page)
      tag :li, link(page, page, :rel => rel_value(page)), :class => ('active' if page == current_page)
    end
 
    def gap
      tag :li, link(super, '#'), :class => 'disabled'
    end
 
    def previous_or_next_page(page, text, classname)
      tag :li, link(text, page || '#'), :class => [classname[0..3], classname, ('disabled' unless page)].join(' ')
    end
  end
 
  def page_navigation_links(pages)
    will_paginate(pages, :class => 'pagination', :inner_window => 2, :outer_window => 0, :renderer => BootstrapLinkRenderer, :previous_label => '←'.html_safe, :next_label => '→'.html_safe)
  end

Deploying Ruby on Rails 3.1 on Ubuntu

Leo — Tue, 27 Dec 2011 07:18:48 +0000

This serves as a reminder for me on getting Ruby on Rails 3.1.3 running on Ubuntu (tested on 10.01 LTS).

Install git using apt-get:

sudo apt-get install git-svn

Install build dependencies:

sudo apt-get install gcc g++ build-essential libssl-dev libreadline5-dev zlib1g-dev linux-headers-generic libsqlite3-dev

Install rvm (Ruby Version Manager)

bash -s stable < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer )

Put RVM into .bash_rc file so that you can run it easily:

echo '[[ -s "$HOME/.rvm/scripts/rvm" ]] && . "$HOME/.rvm/scripts/rvm" # Load RVM function' >> ~/.bash_rc

Reload bash settings:

source ~/.bash_rc

That should install rvm. Now you can use it to install Ruby:

rvm install 1.9.2

Make sure to switch into the correct version of Ruby:

rvm use 1.9.2

Install Rails:

gem install rails

This takes a while….

Install Apache:

sudo apt-get install apache2

Install Phusion Passenger:

gem install passenger

passenger-install-apache2-module

Follow the instructions on screen.

Setup Apache virtual hosting the way you normally do. Make the DocumentRoot the rails public directory.

<VirtualHost *:80>
        ServerName ec2-107-21-154-173.compute-1.amazonaws.com
        RailsEnv production
        DocumentRoot /home/ubuntu/depot/public
        <Directory /home/ubuntu/depot/public>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride All
                Order allow,deny
                allow from all
        Directory>
 
VirtualHost>

When deploying an app make sure to run bundle install to make sure all the dependencies are installed. This includes mysql, etc.

Rails 3.1 needs a Javascript runtime.

Add this to your gemfile:

gem 'execjs'
gem 'therubyracer'

Run:

bundle install

Make sure to setup your DB using:

rake db:setup RAILS_ENV="production"

Restart apache:

sudo service apache2 restart

Load your browser and visit the site. Hopefully it works, if it doesn’t, check logs/production.log and trace from there.

Asset Pipeline Gotchas

The new asset pipeline in Rails may cause your app to fail.

When you deploy an app, make sure that you precompile the assets using:

bundle exec rake assets:precompile

If you are including css files that are not in your application.css manifest, you need to manually include them in your config/environments/production.rb:

config.assets.precompile += ['960.css']

Note: After recompiling assets you will need to restart your rails app. Do this either by restarting apache or creating/modifying “tmp/restart.txt” in your rails app. Passenger will look at the timestamp for restart.txt and restart rails accordingly.

Rcconf – Tool to edit Linux boot services

Leo — Tue, 15 Nov 2011 20:53:07 +0000

I know I’ll forget this because I only have to edit runlevel services once in a blue moon. Use a tool called rcconf. You may have to install it using apt-get.

VBulletin single sign on

Leo — Tue, 01 Nov 2011 16:01:29 +0000

I figure I would write this because I searched around for a suitable solution and it’s quite frustrating to find. VBulletin (the company) runs a very tight community and generally don’t help people who want to hack or extend the software. In my case, I am integrating VBulletin into a client’s custom application and trying to get single sign on to work. Do a search and you’ll find VBulletin telling people who have asked the same question “we won’t help you. Go to vbulletin.org.” When you do get to vbulletin.org, you can’t see any code because you’re not a licensed user. Very frustrating.

So…here it is.

VBulletin stores two cookie vars that can be used for single sign on:

bb_userid – look in the ‘user’ table, it is the field ‘userid’
bb_password – md5(user.password . COOKIE_SALT)

When a user visits the forum, it can login based on the above.

Once you authenticate that the user can login in your app, read the user row from the vbulletin database. Take the password (which is already hashed) and concatenate the COOKIE_SALT constant to it. You can find COOKIE_SALT defined in /includes/functions.php. You then md5 hash this string and set the cookie as bb_password.

When setting your cookie, remember to set the domain to one that vbulletin can read. If you’re using different subdomains (e.g. www.leonardteo.com for the main app and forums.leonardteo.com for the forums) simply set the domain to “.leonardteo.com”. In VBulletin, you’ll want to go into the admincp and change the cookie domain to the same.

By setting these two cookies, you should be able to login with your app and it will automatically login to VBulletin as well.

Logout
Make sure you set the logout routine as well. Your logout routine needs to clear three cookies:

bb_userid
bb_password
bb_sessionhash

Enjoy.

My rant about storing crucial web app settings in a database

Leo — Tue, 01 Nov 2011 12:51:37 +0000

I’ve been coding PHP for a long time. Since 1999 actually. That means that for most of my adult life, I’ve been working with this programming language in the web-o-sphere in general. I’ve gotten to work on many applications and continue to do so. I’ve also gotten to use many off-the-shelf apps including WordPress, Expression Engine and VBulletin. Today, I’d like to rant about two conventions that these all have in common, and why we need to be rid of them.

1. Storing paths and URLs in the database

This happens in WordPress, Expression Engine and VBulletin. Basically, the idea is that all settings must be stored in a database table because…well…it’s supposed to be easier. The user updates a setting in the admin control panel, and it stores those settings in a database. The problem with storing paths in a database like this is that it makes the app less portable. For example, say you want to move an installation of any of the above-said apps to a different domain, it’s not as simple as tarballing the entire application, running mysqldump and restoring on the other side. You’d think that changing the config file would get it to work but this is only half the battle. WordPress and Vbulletin, for example, stores the actual URL of the application in the database, so when it does any redirect (e.g. when you login to the admin control panel), it will redirect to your OLD site.

WordPress does something else that I consider evil. When you upload images, it stores the full web path in the meta data. This is also a problem when you move to another domain because all your images are now pointing to the old domain. I’ve had to write scripts that change all metadata values and wordpress settings when porting from one domain to another. It’s annoying.

Why is this an issue though?

Development. For most people, the above shouldn’t matter because you have a blog that you installed on a webserver; you update it and all’s well. You never run a local copy of the site for development. As a developer though, I have to usually get local and staging copies of the website running. This means that I’m constantly having to do all kinds of shenanigans to get these instances running on different servers.

I used to think that this was simply a bad design limited to a couple of web apps but it seems like a whole plethora of web apps suffer from this. While working for Gnomon, we bought into Expression Engine, thinking that this would be the silver bullet that kills WordPress. Nada. EE is just as BADLY designed! (not just for the reasons above, but many others)

2. Serializing Critical Settings

PHP has a neat function called serialize(). It takes data and turns it into a string. The data can be read back in easily. This is great for many things. For example, if you have user accounts and each user has their own settings, it’s probably useful to just serialize their settings data and save it as a single string in the database.

Crucial site settings though, should NEVER be serialized in a database. I would go as far to say that as a rule of thumb, if the setting is crucial enough to make or break an installation of an app, it should be in a config file. This relates to my first rant as well. Many settings can be put in the database, but some settings (like paths), should just be in flat config files that can be quickly changed when moving the app.

I was absolutely shocked when using Expression Engine that crucial app-breaking settings like upload paths were not only stored in the database but SERIALIZED. I ended up writing a script that can move an entire installation of EE from one server to another in a single command line, and it’s really long because it had to connect to the database, unserialize the data, make modifications, re-serialize the data, write back to database.

It shouldn’t be this hard.

My recommendation

My recommendation is to follow this rule of thumb: if the setting is crucial enough to make or break an installation of an app, it should be in a config file.

Absolute paths like upload paths, etc. should be stored in a config file.

If the application needs a site-wide URL, this should be in a config file.

Individual asset links in the database should link to the relative location (i.e. minus the hostname.) e.g. not http://leonardteo.com/image1.jpg, but /image1.jpg instead. Even better, just store ‘image1.jpg’ and use a configuration setting to determine what the path to that image is.

The litmus test should be: Tarball an application, dump the database. Copy everything to a different computer/hostname. Restore everything. If you can’t get the site up and running again without accessing and changing values in the database, fail.

A practical benefit – fast deployment

On a closing note, one of the benefits of running critical site settings in config files, is that you can switch configurations based on the hostname.

E.g. in your config file:

switch ($_SERVER['SERVER_NAME']){
    case 'cgacontest.local':
        define('SITE_MODE', "LOCAL");
        break;
    case 'cgacontest.test':
        define('SITE_MODE', "TEST");
        break;
    case '3dsmaxdesign.cgarchitect.com':
        define('SITE_MODE', "LIVE");
        break;
}

With the above code, we can switch between site modes and run different configurations automatically based on the hostname. This means that when it comes to deployment, I simply push any changes to the server without having to change configuration.

Deleting files in Unix in 1 command

Leo — Fri, 15 Jul 2011 11:52:45 +0000

I find myself constantly doing this with files that some OS’s and programs spew all over a directory structure. E.g. .DS_Store, .svn, etc.

find . -name "filename" -exec rm -rf {} \;

Taking Server Side Screenshots of Websites

Leo — Wed, 13 Jul 2011 13:01:42 +0000

(updated the post so that it works on Ubuntu 11.10)

I was recently doing some testing to see if it was possible to take server side screenshots of websites. After a bit of fudging around, managed to do it…. Now, I can’t take credit for figuring this out. Other people have done this before me and some have posted blog posts about this, but technology changes so fast that some of the blog posts that outlined how to do this no longer work. I got most of the way from a blog post by Gregory Mazurek.

Tools:

Ubuntu Linux (I used Ubuntu 11.10) – 32 bit (so we can run Flash without any problems)
Xvfb
Firefox
Flash
ImageMagick

Theoretically, it sounds very simple. Xvfb lets you run X windows but rather than rendering to a display (servers might not have a display), it renders to a virtual framebuffer. So…you run Firefox in Xvfb, then take a screenshot of it as it’s running. Simple right? lol…

Step 1 - Install all the stuff you need

sudo apt-get install xvfb
sudo apt-get install firefox
sudo apt-get install adobe-flashplugin //You may need to enable the canonical partner repositories. Uncomment appropriate lines in /etc/apt/sources.list
sudo apt-get install imagemagick

Step 2 – Get the xvfb process running

Note: Credit to Gregory Mazurek for this shell script…

Create a file in /etc/init.d called xvfb. Put this in:

#!/bin/bash
#
# /etc/rc.d/init.d/xvfb
#
# chkconfig: 345 98 90
# description: starts virtual framebuffer process to
# enable server
#
#
#
# Source function library.
#.  /etc/init.d/functions
XVFB_OUTPUT=/tmp/Xvfb.out
XVFB=/usr/bin/X11/Xvfb
XVFB_OPTIONS=":5 -screen 0 1080x1440x24 -fbdir /var/run"
 
start()  {
echo -n "Starting : X Virtual Frame Buffer "
$XVFB $XVFB_OPTIONS &gt;&gt;$XVFB_OUTPUT 2&gt;&amp;1&amp;
RETVAL=$?
echo
return $RETVAL
}
 
stop()   {
echo -n "Shutting down : X Virtual Frame Buffer"
echo
killall Xvfb
echo
return 0
}
 
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status xvfb
;;
restart)
    stop
    start
    ;;
 
*)
echo "Usage: xvfb {start|stop|status|restart}"
exit 1
;;
esac
exit $?

Once you have this shell script in place, you can simply start and stop the xvfb server easily with:
sudo /etc/init.d/xvfb start (or sudo service xvfb start)
sudo /etc/init.d/xvfb stop

Great!

Now to run Firefox…

Step 3 – Firefox and Flash

I’ll assume that you did the apt-get install to get both Firefox and flash-plugin. Once you have those, test the configuration like this…

1. Make sure that the xvfb server is running
2. Run this:

DISPLAY=:5 nohup firefox http://www.youtube.com &amp;

DISPLAY=:5 // This tells xvfb to render to display 5 (virtual)
nohup //silence the output
firefox //loads firefox
http://youtube.com //loads youtube which tells you whether you have Flash or not
& //Load this in the background

Firefox should be running in the background now. If it’s not, you may have to debug and find out why. Remove the nohup and the & to see the output to shell if you need to debug.

Now comes the moment of truth. Taking a screenshot:

DISPLAY=:5 import -window root screenshot.png

This will dump the desktop to screenshot.png. Check it.

Kill firefox by typing ‘fg’ then CTRL+C.

Step 4 – Firefox configuration

We have to override the default Firefox configuration settings. E.g. it has one that will try to resume a crash.

Firefox stores its preferences in the user root ~/.mozilla. On each system and each user it’s different. Look in: ~/.mozilla/firefox/

In there, you should see a directory with funny numbers and letters followed by a .default. Mine is 6mqvagz4.default. Chdir into there. Create a file called user.js.

The problem here is that we either have to edit files manually or have to get Firefox to generate the appropriate settings for us. If you are already using Ubuntu at home, this is much simpler because you can set up Firefox however you like, then take the entire preferences directory and copy its contents into the server’s. This is what I ended up doing. That route is by far the easiest and recommended.

Otherwise:

In prefs.js, edit these:

pref("browser.startup.page",0);
pref("browser.sessionstore.resume_from_crash", false);

Now…another annoyance is that the Firefox window may not be at the size you want it. So you need to edit the file ‘localstore.rdf’. What I ended up doing was running Firefox on my Mac at home, deleting the localstore.rdf file, having Firefox generate a new one. On quitting, it saves it out. This is my localstore.rdf file that I ended up using:

localstore.rdf:

Notice the width and height of the window. I set it to this portrait width/height so that I could grab square images.

Step 5 – Further Automation

From here, you can do further automation to make the process of taking a screenshot really simple. I created this script:

#!/bin/bash
#screengrab
 
FIREFOX_OUTPUT=/tmp/firefox.out
TIMESTAMP=$(date +%s)
DELAY=5
 
#Check for args
if [ $# -ne 1 ]
        then
                echo "You need to pass a URL in. E.g. urlscreengrab www.google.com"
                exit
fi
 
#Calculate how many instances of grab we have running
COUNT=$(ps -e | grep -c 'grab.sh')
COUNT=$((COUNT-2))
#I don't know why but the above returns 2 when there are 0 processes running.
#if grab.sh is already running, wait for it to finish
WAIT=$((DELAY*COUNT))
if [ "$WAIT" -gt "0" ]
        then
                WAIT=$((WAIT+2)) #Add 2 seconds to wait for other grab processes to finish
fi
echo "Waiting $WAIT seconds for other grab processes to finish"
sleep $WAIT
 
#Load up firefox
echo "Loading Firefox"
DISPLAY=:5 firefox $1 &gt;&gt;$FIREFOX_OUTPUT 2&gt;&amp;1&amp;
FFPID=$!
echo "Standby...waiting $DELAY seconds for browser to load..."
sleep $DELAY
DISPLAY=:5 import -window root -crop 1000x1000+4+86 $TIMESTAMP.png
#DISPLAY=:5 import -window root $TIMESTAMP.png
#killall firefox-bin
kill $FFPID
echo "$TIMESTAMP.png"

To use this script, type: ./urlscreengrab http://www.youtube.com

With the above configuration, it will automatically take the screenshot and crop it into a 1000×1000 square png file. It also has process checking, so it will check for existing Firefox processes and wait until the other grab processes finish. It will also wait 5 seconds for a web page to finish loading before taking the shot. This is needed on many web pages because it takes time to load the page.

Step 6 – Hook up to your web scripts

From here, I just call the web script from PHP using the exec() command. Make sure that Apache is running using the same user as you set up your above scripts so that it can use the correct Mozilla profile. To check, create a php script with the following:

echo exec('whoami');

This should give you the user of the owner of the apache/php process.

My web script for grabbing the image:

/*
Script for grabbing a website image
*/
 
if (!isset($_GET['url']))
{
        echo "No URL provided";
        exit();
}
 
//chdir into the correct dir
chdir('images');
 
//Grab the site
$exec = "/home/ubuntu/grab.sh ".$_GET['url'];
echo "Executing: ".$exec."";
$lastline = exec($exec, $out, $status);
 
foreach ($out as $line)
{
        echo $line."";
}
echo "Finished with status: $status ";
 
if ($status == 0)
{
        echo "&quot;images/$lastline\&quot;">Your image is here";
}

There you go…have fun.

Quick and dirty WordPress profiling

Leo — Wed, 13 Jul 2011 11:59:08 +0000

Recently while working on one of our client sites CG Channel, the front page was taking around 8-10 seconds to load. This was driving me crazy. Somehow, the page had gotten so bloated that it was taking what the web considers to be an eternity to show anything on screen. At first, I assumed it was the RSS feeds that it was reading to update various content on the site, but soon found that information was cached. What now?

Profiling to the rescue. In PHP, you have some pretty good tools that enable you to do profiling on your application. In the case of WordPress which is mostly theme (view) driven, it’s almost easier because all the processing is called from the view itself. In the case of CG Channel homepage home.php, I put the following code in…

This goes at the top of the page:

$timer = FALSE;
 
if ($timer)
{
	//Timing functions
	$time_start = microtime(true);
	$global_time_start = $time_start;
}

It basically sets up a timer. The $timer variable is for you to turn the timer on or off. If you put lot’s of timers in, you can simply switch it off later rather than having to remove all the code.

The PHP microtime() function returns the current unix timestamp with microseconds. Just what we need.

$time_start holds the time for the current block.
$global_time_start holds the time for the entire page.

To time a previously rendered section, you just do this:

if ($timer)
{
	$time = microtime(true) - $time_start;
	echo "\"color:red\">$time seconds";
	$time_start = microtime(true);	
}

So…for example….:

$timer = FALSE;
 
if ($timer)
{
	//Timing functions
	$time_start = microtime(true);
	$global_time_start = $time_start;
}
 
get_header();
 
if ($timer)
{
	$time = microtime(true) - $time_start;
	echo "\"color:red\">$time seconds";
	$time_start = microtime(true);	
}

This would time the get_header() section of the site. When you refresh the page, it will show the time it’s taken to render the previous section.

At the bottom of the page, you can put:

if ($timer)
{
	$time = microtime(true) - $time_start;
	echo "\"color:red\">$time seconds";
	$time_start = microtime(true);		
	$global_time = $time_start - $global_time_start;
	echo "\"color:red\">Total page render time: $global_time seconds";
}

This will give you a time for the previous block that was rendered, and the global time that it took for the whole page to run.

Doing this, I found the culprit. Using WP_Query() to fetch posts and using the wordpress loop was very slow: about 4 seconds for fetching and displaying 12 news items. I found another function get_posts() which returned all the information I needed to show the news items, so I switched to using that. Rather than using a WordPress loop, I used a simple foreach() loop. This brought each query down to under 1 second!

To further optimize, rather than using WordPress template tags to directly output data, I cached them into variables. For example, if you are going to call get_post_meta() on the same meta information multiple times in a loop, save the data into a variable and use the variable instead. This saves WordPress the trouble of doing unnecessary database calls.

//Return array of posts
$posts = get_posts(array('category' => 1, 'numberposts' => 12));
 
//If you want to see what's in the array...
//var_dump($posts);
 
//Loop through each post
foreach ($posts as $post):
 
	//Cache the variables
	$permalink = get_permalink($post->ID);
	$thumbnail_b = get_post_meta($post->ID, "thumbnail_b", true);
	$thumbnail_s = get_post_meta($post->ID, "thumbnail_s", true);
	$plug_title = get_post_meta($post->ID, "plug_title", true);
 
	//Do whatever output you want here....
 
endforeach;

I did some further optimization of the site with minification of javascript, getting rid of unnecessary code, etc. My efforts resulted in the site going from taking about 10 seconds to load, down to 1-1.5 seconds. That’s not bad. Could be faster, but at least the site feels snappy now!

Running Nginx and PHP5 on Ubuntu

Leo — Fri, 08 Jul 2011 22:44:21 +0000

Very simple to set up Nginx, PHP5 and MySQL on Ubuntu. Hats off to the Ubuntu guys for making it so simple.

sudo apt-get install nginx
sudo apt-get install php5-cgi (you might need to also need to install php5)

Test that nginx works by opening your browser and going to http://localhost/

Edit /etc/nginx/sites-available/default

Search for the bits that say “# pass the PHP scripts to FastCGI server….” uncomment those configuration bits.

Create index.php in /usr/share/nginx/www/

In it, put:

In a web browser go to http://localhost/index.php It should give you a “bad gateway error”. This is because php fastCGI isn’t running. Go back to your terminal and type “php5-cgi -b 127.0.0.1:9000 &” This should start php fastCGI in the background. Now visit http://localhost/index.php again and you should see the php info screen.