Benchmarking HipHopVM against PHP 5.3 on Ubuntu 12.04

HipHopVM is Facebook’s high-performance PHP runtime, that is built to be a production-ready virtual machine that delivers superior performance. Facebook was using a PHP-to-C++ compiler (HPHPc), but is now moving to HipHopVM (HHVM) to unify their production and development environments.

HipHopVM Performance

PHP 5.3.10 is the current version of PHP on Ubuntu 12.04 (the latest LTS release). On Ubuntu 12.04 HipHopVM has binary packages available (v2.1.0).


This graph shows the average execution time (of 10 runs) on both PHP 5.3.10 (with APC) and HipHopVM v2.1.0. The Y-axis shows the amount of seconds the execution took (on average) and the numbers on the X-axis represent the following benchmark scripts:

  1. bench1.php:
  2. bench2.php:
  3. bench3.php:
  4. bench4.php:
  5. bench5.php:
  6. bench6.php:

This is the raw data:


Note that the memory usage (in bench3.php) is also different:



The differences were smaller and less consistent than I expected. Also I was disappointed that HipHopVM did not run my code.

NB: HHVM v2.2.0 is out and it runs Symfony!

What do you think about these numbers? Please share your thoughts in the comments below.

12 thoughts on “Benchmarking HipHopVM against PHP 5.3 on Ubuntu 12.04”

  1. @Isaac: Yes, valid point, but the problem is that not all code runs in HipHopVM. If it did I would have benchmarked on the Symfony or CakePHP framework.

  2. @Yuri: Interesting link, thank you. I am hopefully running the code often to have enough warmup for optimal performance. If you have suggestions that improve the benchmark, please share.

  3. 1) These are the tasks from bench6.php that were rejected from the benchmarks game back in 2005 because it was so obvious they could easily be replaced by tasks that made some kind-of sense —

    ackermann, ary, ary2, fibo, hash1, hash2, heapsort, matrix, nestedloop, sieve, strcat

    2) If Symfony or Cake won’t run on HipHop then your challenge is to find something real that will.

    The benchmarks game website is PHP —

    The only framework used is 200 lines of Brian Lozier’s templating code —

    No one will ever think the code is great PHP but at least it’s generating XHTML from data.

  4. The benchmark scripts used all have one major flaw: They disable the JIT.

    Take a look at where Owen explains why global-scope code is not JITted (and can’t be, really) to machine code and thus will only be about on-par with PHP. If you just wrap these benchmarks in `function f() { /* benchmark */ } f();` you’ll see a speed up. Also, bear in mind that the JIT isn’t enabled by default for command-line scripts (because perf isn’t the main goal there), you should explicitly turn it on with:

    hhvm -vEval.Jit=1 bench.php

  5. @Sara: Thank you for your comment, very informative. FYI, I did not run the scripts from the command-line. I will try the function scope around the benchmark scripts to see whether it makes a significant difference.

  6. @AntonBe: Yes, I am already setting it up for tests. Since it promised to allow for Symfony to run as well, it takes me a little more time. It will be one of the next posts.

  7. @Maurits, I’m excited about that as well! That also hopefully means Laravel compatibility, which I like a bit more than Symfony for various reasons.

  8. What are you trying to bechmark, exactly? The first 11 runs in HHVM are to warm up the JIT, which means that they are going to be significantly slower than they would usually be. You should start benchmarking from the 12th run.

    It’s a little bit unfortunate that this post comes at the top when searching for hhvm and php speed comparisons.

  9. @Adri: Thank you for your comment and your concern. I took the obvious problem of warming up the cache into account and did a proper warmup. Did you find any significant different results that you want to share?

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>