A couple of years ago now, I rather blithely explained how you can get a working Behat test suite, to implement behavioural-driven development, in under ten minutes. That's all still true, but it's very much in the context of Javascript-free, CSS-free browsing: basically, testing what a robot might see if it visited your website. While that can test a lot of functionality, there's also key elements to any website that can't be tested without seeing how they look and behave in a "real.html" browser.
In theory, Behat's documentation makes Javascript support seem simple: just tag your scenario with "@javascript". But all of this assumes a technical stack behind every test, that you've already set up. There are a number of different ways of wiring up Behat to such a "real" browser, but often other requirements prevent one or some of them from being used.
Below is a summary of some of the choices I had to make, to get Javascript tests working in my own environment. I'm not presenting it as a definitive "howto", because I'm sure if I did it again, I'd do some bits of it differently: but it should provide some examples of problems you might run into, and some diagnostics and workarounds that might help you proceed.
If your hardware (emulated or otherwise) lacks a graphical display
For example, my own development and testing all happens in a virtual machine (VM) managed by Vagrant and Virtualbox. This VM doesn't have a screen as such: it's "headless", meaning it not only doesn't have an emulated monitor, but also (without further installation and configuration) has no software that might manage even a virtual "fake" display. This means that, even if a browser as a user might understand it could be fired up, and then led through a set of tasks, it might be incapable of carrying out those tasks and reporting back, because the website has nowhere to be rendered.
Under Linux, the longstanding monitor control system of choice is X (Windows). A headless version of this exists, to "fake" a display, called X Virtual Framebuffer or Xvfb. The choice of browser for testing can also determine other library requirements, as we'll see below.
(If you're not running tests headlessly, but just on your own local laptop, integration is much more straightforward!)
If your operating system is already specified
My current project has specific hosting requirements, which in turn limit how the VM has to behave. This means the VM must run Debian Jessie. As I'd already decided to try testing using the Firefox browser, its most recent versions were not compatible with Jessie's pre-built packages for GTK 3.0. This limitation meant that the maximum Firefox version that would work was 45, which I found was most stable under GTK 2.x.
If your browser version is now specified
With a particular browser version in mind, Behat needs to connect to it. I use Selenium, a Java application (and hence need a JRE too). But the Selenium version needs to be compatible with the Firefox version, so it can talk to the correct Firefox APIs: otherwise, you'll get errors along the lines of:
Unable to connect to host 127.0.0.1 on port 7055 after 45000 ms.
Followed by a long debug output. This is a signal that Selenium needs to be a different version (usually an older one, if like me you're on Firefox 45!) But at this late stage, different Selenium JARs can be evaluated by trial and error.
The result
Other combinations of packages might also work, but here's one working set of package versions that does:
- Operating system: Debian Jessie
- Graphical emulation: Xvfb, libgtk2.0-0, libasound2 (Firefox needs virtual audio too)
- Behat-to-browser connections: Selenium 2.53.1, OpenJDK 7 JRE (headless)
Automated provisioning with Ansible
I provision my VMs with Ansible where possible, so here's an Ansible tasks file describing the above:
- name: Get Firefox headless, OpenJDK JRE and XVFB
apt: "name={{ item }} state=latest"
with_items:
- openjdk-7-jre-headless
- xvfb
- libgtk2.0-0
- libasound2
- name: Download Firefox at a particular version compatible with gtk<3.2 and Selenium 2.53.1.
unarchive:
src: https://ftp.mozilla.org/pub/firefox/releases/45.0.1esr/linux-x86_64/en-GB/firefox-45.0.1esr.tar.bz2
dest: /opt
remote_src: true
- name: Symlink Firefox executable.
file: src=/opt/firefox/firefox dest=/usr/bin/firefox state=link
- name: Create folder to contain Selenium jar.
file: path=/opt/selenium state=directory mode=0755
- name: Download Selenium jar compatible with Firefox 45.
shell: |
wget -O /opt/selenium/selenium.jar http://selenium-release.storage.googleapis.com/2.53/selenium-server-standalone-2.53.1.jar
args:
creates: /opt/selenium/selenium.jar
# @see https://blog.vandenbrand.org/2014/11/05/install-selenium-headless-on-debian-wheezy-optionally-with-ansible/
- name: Create selenium as a service.
copy: src=templates/etc-init.d-selenium.j2 dest=/etc/init.d/selenium mode=0755
- name: Start Selenium as a service.
shell: /etc/init.d/selenium restartYou also need Xvfb to be started as a service, so it's already running by the time the Behat tests run. I adapted this example init script to emulate a much bigger monitor, so that I can test behaviours on desktop-sized displays:
### BEGIN INIT INFO
# Provides: selenium
# Required-Start: $local_fs $network
# Required-Stop: $local_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: selenium
# Description: selenium with xvfb
### END INIT INFO
XVFB="/usr/bin/xvfb-run"
PIDFILE=/var/run/xvfb.pid
case "$1" in
start)
echo -n "Starting virtual X frame buffer: Xvfb"
start-stop-daemon --start --quiet --pidfile $PIDFILE --make-pidfile --background --exec $XVFB -- -s '-screen 0 1280x1024x8' java -jar /opt/selenium/selenium.jar
echo "."
;;
stop)
echo -n "Stopping virtual X frame buffer: Xvfb"
# does not work with start-stop-daemon because of child processes
read pid <$PIDFILE
pkill -TERM -P $pid
echo "."
;;
restart)
$0 stop
$0 start
;;
*)
echo "Usage: /etc/init.d/selenium {start|stop|restart}"
exit 1
esac
exit 0
This can be saved in a templates folder alongside the Ansible YAML file in the tasks folder.
Summary
Behat Javascript integration, once it's set up, is very straightforward and stable. But depending on the limitations of your existing environment, the chain of Behat–Selenium–browser–display can be very picky about the versions of each element of the chain. Above is a recipe that works, although others might work too.
Ultimately the increase in test coverage is definitely worth it—it shook out some bugs for us that weren't testable any other way!—but expect it to take some time to get it working first.
