(updated the post so that it works on Ubuntu 11.10)
I was recently doing some testing to see if it was possible to take server side screenshots of websites. After a bit of fudging around, managed to do it…. Now, I can’t take credit for figuring this out. Other people have done this before me and some have posted blog posts about this, but technology changes so fast that some of the blog posts that outlined how to do this no longer work. I got most of the way from a blog post by Gregory Mazurek.
Tools:
- Ubuntu Linux (I used Ubuntu 11.10) – 32 bit (so we can run Flash without any problems)
- Xvfb
- Firefox
- Flash
- ImageMagick
Theoretically, it sounds very simple. Xvfb lets you run X windows but rather than rendering to a display (servers might not have a display), it renders to a virtual framebuffer. So…you run Firefox in Xvfb, then take a screenshot of it as it’s running. Simple right? lol…
Step 1 - Install all the stuff you need
sudo apt-get install xvfb
sudo apt-get install firefox
sudo apt-get install adobe-flashplugin //You may need to enable the canonical partner repositories. Uncomment appropriate lines in /etc/apt/sources.list
sudo apt-get install imagemagick
Step 2 – Get the xvfb process running
Note: Credit to Gregory Mazurek for this shell script…
Create a file in /etc/init.d called xvfb. Put this in:
#!/bin/bash
#
# /etc/rc.d/init.d/xvfb
#
# chkconfig: 345 98 90
# description: starts virtual framebuffer process to
# enable server
#
#
#
# Source function library.
#. /etc/init.d/functions
XVFB_OUTPUT=/tmp/Xvfb.out
XVFB=/usr/bin/X11/Xvfb
XVFB_OPTIONS=":5 -screen 0 1080x1440x24 -fbdir /var/run"
start() {
echo -n "Starting : X Virtual Frame Buffer "
$XVFB $XVFB_OPTIONS >>$XVFB_OUTPUT 2>&1&
RETVAL=$?
echo
return $RETVAL
}
stop() {
echo -n "Shutting down : X Virtual Frame Buffer"
echo
killall Xvfb
echo
return 0
}
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status xvfb
;;
restart)
stop
start
;;
*)
echo "Usage: xvfb {start|stop|status|restart}"
exit 1
;;
esac
exit $?
Once you have this shell script in place, you can simply start and stop the xvfb server easily with:
sudo /etc/init.d/xvfb start (or sudo service xvfb start)
sudo /etc/init.d/xvfb stop
Great!
Now to run Firefox…
Step 3 – Firefox and Flash
I’ll assume that you did the apt-get install to get both Firefox and flash-plugin. Once you have those, test the configuration like this…
1. Make sure that the xvfb server is running
2. Run this:
DISPLAY=:5 nohup firefox http://www.youtube.com &
DISPLAY=:5 // This tells xvfb to render to display 5 (virtual)
nohup //silence the output
firefox //loads firefox
http://youtube.com //loads youtube which tells you whether you have Flash or not
& //Load this in the background
Firefox should be running in the background now. If it’s not, you may have to debug and find out why. Remove the nohup and the & to see the output to shell if you need to debug.
Now comes the moment of truth. Taking a screenshot:
DISPLAY=:5 import -window root screenshot.png
This will dump the desktop to screenshot.png. Check it.
Kill firefox by typing ‘fg’ then CTRL+C.
Step 4 – Firefox configuration
We have to override the default Firefox configuration settings. E.g. it has one that will try to resume a crash.
Firefox stores its preferences in the user root ~/.mozilla. On each system and each user it’s different. Look in: ~/.mozilla/firefox/
In there, you should see a directory with funny numbers and letters followed by a .default. Mine is 6mqvagz4.default. Chdir into there. Create a file called user.js.
The problem here is that we either have to edit files manually or have to get Firefox to generate the appropriate settings for us. If you are already using Ubuntu at home, this is much simpler because you can set up Firefox however you like, then take the entire preferences directory and copy its contents into the server’s. This is what I ended up doing. That route is by far the easiest and recommended.
Otherwise:
In prefs.js, edit these:
pref("browser.startup.page",0);
pref("browser.sessionstore.resume_from_crash", false);
Now…another annoyance is that the Firefox window may not be at the size you want it. So you need to edit the file ‘localstore.rdf’. What I ended up doing was running Firefox on my Mac at home, deleting the localstore.rdf file, having Firefox generate a new one. On quitting, it saves it out. This is my localstore.rdf file that I ended up using:
localstore.rdf:
<?xml version="1.0"?>
<RDF:RDF xmlns:NC="http://home.netscape.com/NC-rdf#"
xmlns:RDF="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<RDF:Description RDF:about="about:config#lockCol"
ordinal="3" />
<RDF:Description RDF:about="about:config#prefCol"
ordinal="1"
sortDirection="ascending" />
<RDF:Description RDF:about="about:config#valueCol"
ordinal="7" />
<RDF:Description RDF:about="chrome://browser/content/browser.xul#sidebar-title"
value="" />
<RDF:Description RDF:about="about:config#typeCol"
ordinal="5" />
<RDF:Description RDF:about="chrome://browser/content/browser.xul">
<NC:persist RDF:resource="chrome://browser/content/browser.xul#main-window"/>
<NC:persist RDF:resource="chrome://browser/content/browser.xul#sidebar-box"/>
<NC:persist RDF:resource="chrome://browser/content/browser.xul#sidebar-title"/>
</RDF:Description>
<RDF:Description RDF:about="chrome://browser/content/browser.xul#main-window"
screenY="0"
width="1080"
screenX="0"
height="1440"
sizemode="maximized" />
<RDF:Description RDF:about="about:config">
<NC:persist RDF:resource="about:config#prefCol"/>
<NC:persist RDF:resource="about:config#lockCol"/>
<NC:persist RDF:resource="about:config#typeCol"/>
<NC:persist RDF:resource="about:config#valueCol"/>
</RDF:Description>
</RDF:RDF>
Notice the width and height of the window. I set it to this portrait width/height so that I could grab square images.
Step 5 – Further Automation
From here, you can do further automation to make the process of taking a screenshot really simple. I created this script:
#!/bin/bash
#screengrab
FIREFOX_OUTPUT=/tmp/firefox.out
TIMESTAMP=$(date +%s)
DELAY=5
#Check for args
if [ $# -ne 1 ]
then
echo "You need to pass a URL in. E.g. urlscreengrab www.google.com"
exit
fi
#Calculate how many instances of grab we have running
COUNT=$(ps -e | grep -c 'grab.sh')
COUNT=$((COUNT-2))
#I don't know why but the above returns 2 when there are 0 processes running.
#if grab.sh is already running, wait for it to finish
WAIT=$((DELAY*COUNT))
if [ "$WAIT" -gt "0" ]
then
WAIT=$((WAIT+2)) #Add 2 seconds to wait for other grab processes to finish
fi
echo "Waiting $WAIT seconds for other grab processes to finish"
sleep $WAIT
#Load up firefox
echo "Loading Firefox"
DISPLAY=:5 firefox $1 >>$FIREFOX_OUTPUT 2>&1&
FFPID=$!
echo "Standby...waiting $DELAY seconds for browser to load..."
sleep $DELAY
DISPLAY=:5 import -window root -crop 1000x1000+4+86 $TIMESTAMP.png
#DISPLAY=:5 import -window root $TIMESTAMP.png
#killall firefox-bin
kill $FFPID
echo "$TIMESTAMP.png"
To use this script, type: ./urlscreengrab http://www.youtube.com
With the above configuration, it will automatically take the screenshot and crop it into a 1000×1000 square png file. It also has process checking, so it will check for existing Firefox processes and wait until the other grab processes finish. It will also wait 5 seconds for a web page to finish loading before taking the shot. This is needed on many web pages because it takes time to load the page.
Step 6 – Hook up to your web scripts
From here, I just call the web script from PHP using the exec() command. Make sure that Apache is running using the same user as you set up your above scripts so that it can use the correct Mozilla profile. To check, create a php script with the following:
This should give you the user of the owner of the apache/php process.
My web script for grabbing the image:
/*
Script for grabbing a website image
*/
if (!isset($_GET['url']))
{
echo "No URL provided";
exit();
}
//chdir into the correct dir
chdir('images');
//Grab the site
$exec = "/home/ubuntu/grab.sh ".$_GET['url'];
echo "Executing: ".$exec."";
$lastline = exec($exec, $out, $status);
foreach ($out as $line)
{
echo $line."";
}
echo "Finished with status: $status ";
if ($status == 0)
{
echo "<a href="\"images/$lastline\"">Your image is here</a>";
}
There you go…have fun.