Posts Tagged ‘php’

Symfony2 Guzzle bundle for cURL API calling

The LswGuzzleBundle adds Guzzle API call functionality to your Symfony2 application. It is easy to use from the code and is aimed to provide full debugging capabilities. source: Github

On Packagist, you can see that in the past two years we have published 10 LeaseWeb Symfony2 bundles. The latest and tenth addition to the list is the LswGuzzleBundle. It is an alternative for the LswApiCallerBundle, so it also is a bundle that helps you to call API’s using cURL. The difference is that this bundle uses Guzzle as a wrapper for cURL.

Guzzle profiler and debugger

This bundle also adds a debug panel to your Symfony2, where you can find the cURL API calls Guzzle makes in your application. It provides profiling and extensive debugging capabilities to your Symfony2 application. We feel it is an essential bundle if you create an application that does not work with a database, but instead has an API as data store backend. Guzzle allows you to specify the parameters and and available calls in a Guzzle service description. Since this allows the bundle to understand the structure of the call, it will display the parsed parameters and output. This is an unique feature of this bundle.

guzzle_bundle

Open source

The bundle is MIT licensed. You can find it at:

As always there is a readme documentation on GitHub to get you started.

“HTTP secure link” using OpenResty Nginx module

Whenever you generate a link on your website to let your visitor download some content, you run into the risk that people will start sharing that link with others. If you want to discourage that you can require the user to login before downloading. If this is not an option for you then you can generate links that are only valid for a few seconds. This is a feature that Nginx supports using the Nginx module HttpSecureLinkModule.

Standard HttpSecureLinkModule

You can specify a Unix timestamp on which the link should no longer be valid; you add this timestamp to the query string like this:

http://localhost/package.zip?e=1402056061&s=be0c677337fec22820b916e48c54755e

In this URL (that uses ‘GET’ parameters) the ‘e’ parameter is the expiry timestamp and the ‘s’ parameter is the signature. This signature is calculated by taking a secret and appending the expiry timestamp to it and calculating a hash over that.

$signature = md5( $secret . $path . $expiry );

OpenResty Nginx modules

But what if you want to use a variant on that? For instance, if you want to restrict on certain networks or use a different hashing algorithm for the signature? My idea was to use the excellent building blocks from the set-misc-nginx-module that is made by Yichun Zhang (a.k.a. “agentzh”). This Nginx module is not included in Nginx by default, but is a part of the OpenResty Nginx distribution or as Yichun Zhang explaines:

By taking advantage of various well-designed Nginx modules, OpenResty effectively turns the nginx server into a powerful web app server. OpenResty is not an Nginx fork. It is just a software bundle. – openresty.org

We did 3 pull requests to the “set-misc-nginx-module” that allow for the needed functionality. You can compile this Nginx module into your Nginx by pointing the “–add-module” configure option to your checkout of this module. We have added the “set_encode_base64url”, “set_decode_base64url”, “set_expired” and “set_ip_matches” directives for building a custom secure link Nginx module.

Installation instructions

The configuration for the Nginx module would be like this:

# GET /?e=1893452400&n=MC4wLjAuMC8w&s=QVvNrcn-j4smvOFhuPTUU7EgoI8
# returns 403 when signature on the arguments is not correct OR
# when expire time is passed or network does not match.
# It is an alternative to the HttpSecureLinkModule in Nginx.
# This example has expiry "2030-01-01" and network "0.0.0.0/0".
location / {
  set_hmac_sha1 $signature 'secret-key' "e=$arg_e&n=$arg_n";
  set_encode_base64url $signature;
  if ($signature != $arg_s) {
    return 403;
  }
  set_expired $expired $arg_e;
  if ($expired) {
    return 403;
  }
  set_decode_base64url $network $arg_n;
  set_ip_matches $ip_matches $network $remote_addr;
  if ($ip_matches = 0) {
    return 403;
  }
  root /var/www;
}

Where the PHP code to generate the link would be:

<?php
function base64url_encode($s)
{ return rtrim(strtr(base64_encode($s),"+/", "-_"),"=");
}
$secret = "secret-key";
$expiry = strtotime("2030-01-01 00:00:00");
$network = base64url_encode("0.0.0.0/0");
$query = "e=$expiry&n=$network";
$signature = base64url_encode(hash_hmac('sha1',$query,$secret,true));
$url = "http://localhost/?$query&s=$signature";
echo "$url\n";

Output would be:

http://localhost/package.zip?e=1893452400&n=MC4wLjAuMC8w&s=QVvNrcn-j4smvOFhuPTUU7EgoI8

Applying a SHA 256 HMAC to an AES 256 cipher in PHP

In a previous post we have looked at encrypting data using AES in PHP. Today we look at how to ensure integrity of the message when encrypting. Doing AES one could simply send the following over the wire (using a randomized IV for every message):

Message = IV || AES-256-Encrypt(Plain-Text)

But this scheme does not allow you to see if there is a corruption in the message. Only if the plaintext is human language, you can see that it does not make sense. But then you have to rely on meta-knowledge about the structure of the plaintext. If you want to automatically detect tampering you need to add a cryptographic hash. A naive way of doing this would be to add a plain SHA-256 like this:

Message = IV || AES-256-Encrypt(Plain-Text || SHA-256(Plain-Text))

But if you have read a little bit about cryptography, you would probably know that you should be using a MAC (Message Authentication Code). MAC algorithms require two parties to share a secret key. MAC algorithms based on cryptographic hash functions (like SHA-256) are called HMAC and are defined in rfc2104. In PHP you can calculate a HMAC using the function “hash_hmac“. It would be convenient, but wrong, to use the same key for the AES and the HMAC and do:

Message = IV || AES-256-Encrypt(Plain-Text || SHA-256-HMAC(Plain-Text))

In the above scheme you take then plaintext and calculate a SHA-256-HMAC over it. You add the result to the end of the message. You encrypt the entire thing with AES-256 using a random IV that you prepend to the message.

When using AES in CBC (Cipher-block chaining) mode each ciphertext block depends on all plaintext blocks processed up to that point. That is why you also need a random IV (Initialization vector) for the first block.

If you tamper with data it has an effect on all the next blocks. When adding a HMAC with the size of a block to the end of the message one would think that this would provide good security. Or as Nick is saying on Scientopia:

Couldn’t you simply append a secure hash of the preceding plaintext to each message? Unless the attacker can predict the effect of his changes (which shouldn’t be possible), he shouldn’t be able to modify the message _and_ the hash in such a way that it remains undetectable, right?

On the other hand Nate from Root Labs is saying:

For schemes like CBC where there is some redundancy, there are a number of cases where the propagation behavior of CBC does not prevent an attack.

I am not a cryptographer, so I cannot argue why the above schemes are wrong (this article might). I did learn that the correct way of applying the HMAC is in fact very simple: First you apply the AES encryption and then you apply the HMAC independently on the IV and crypted message to ensure neither has been tampered with. It looks like this:

Cipher-Text = IV || AES-256-Encrypt(Plain-Text)
Message = Cipher-Text || SHA-256-HMAC(Cipher-Text)

This scheme has the downside that you will need two keys: one for the HMAC and one for the AES encryption. Note that those keys should be unrelated. To illustrate how the correct scheme works I implemented it (in PHP) below:

<?php
// define your key,use length 32,48 or 64 bytes for AES-128,AES-192 and AES-256
$key1 = pack("H*","7206ea04845d1f740401a6e6b2e52d22ac2d314a3d138eb8f61ebe5fb1d6aa5d");
// define the key for the SHA-256 HMAC (both keys should be random shared secrets)
$key2 = pack("H*","dee5cb4a2c4b5b61a729337fc9153daf6387a54b765877752ca36b44a721f469");

// hash comparison function to avoid timing (side channel) attacks
function hash_equals($a, $b) {
  if (!is_string($a) || !is_string($b)) return false;
  if (strlen($a) !== strlen($b)) return false;
  $match = 0;
  for ($i = 0; $i < strlen($a); $i++) {
    $match |= ord($a[$i]) ^ ord($b[$i]);
  }
  return $match===0;
}

// define the plain text
$text = "123";
// get the block size for the padding
$block_size = mcrypt_get_block_size(MCRYPT_RIJNDAEL_128,MCRYPT_MODE_CBC);
// get the length of the padding needed
$pad = $block_size - (strlen($text) % $block_size);
// pad the plain text with the padding character using PKCS7 padding
$text .= str_repeat(chr($pad),$pad);
// get the size of the initialization vector
$iv_size = mcrypt_get_iv_size(MCRYPT_RIJNDAEL_128,MCRYPT_MODE_CBC);
// create a random initialization vector
$iv = mcrypt_create_iv($iv_size,MCRYPT_DEV_URANDOM);
// encode the plain text to get the cipher text
$crypt = mcrypt_encrypt(MCRYPT_RIJNDAEL_128,$key1,$text,MCRYPT_MODE_CBC,$iv);
// calculate the SHA-256 HMAC over the initialization vector + cipher text
$hmac = hash_hmac("sha256",$iv.$crypt,$key2,true);
// add the SHA-256 HMAC to the message
$output = bin2hex($iv.$crypt.$hmac);
// print output displaying the process
echo "$text\n";

// tamper with the data
echo "$output\n";
$output = bin2hex(chr(ord(pack("H*",substr($output,0,2))) ^ 1<<3)).substr($output,2);
echo "$output\n";

// read the data back
$message = pack("H*",$output);
// get the size of the initialization vector
$iv_size = mcrypt_get_iv_size(MCRYPT_RIJNDAEL_128,MCRYPT_MODE_CBC);
// extract the initialization vector from the message
$iv = substr($message,0,$iv_size);
// get the size of the SHA-256 HMAC
$hmac_size = strlen(hash_hmac("sha256","","",true));
// extract the SHA-256 HMAC from the message
$hmac = substr($message,-$hmac_size);
// extract the cipher text from the message
$crypt = substr($message,$iv_size,-$hmac_size);
// calculate the correct SHA-256 HMAC
$crypt_hmac = hash_hmac("sha256",$iv.$crypt,$key2,true);
// check whether the HMACs are equal
if (!hash_equals($hmac,$crypt_hmac)) die("HMAC ERROR\n");
// decode the cipher text to get the plain text
$text = mcrypt_decrypt(MCRYPT_RIJNDAEL_128,$key1,$crypt,MCRYPT_MODE_CBC,$iv);
// get the length of the plain text padding
$pad = ord($text[strlen($text)-1]);
// remove the PKCS7 padding from the plain text
$text = substr($text,0,-1*$pad);
// print output displaying the process
echo "$text\n";

The output of the above code is:

123
99be22775106ca812c432d51de1322fb9ae94362155e921bd383401952a378bf700a90a741f6c037a26a9c8b6dddfe64f684f1b8766a692382d7171ac11a4d73
91be22775106ca812c432d51de1322fb9ae94362155e921bd383401952a378bf700a90a741f6c037a26a9c8b6dddfe64f684f1b8766a692382d7171ac11a4d73
HMAC ERROR

Or if you comment out the tamper line (41):

123
8908cfe140f5ea8a72645147ef183ff9703c7f1bc0f49456a8412e088f4caf0787944d7c54596836491bb6de7a87af6167f99c44729fe03cddaa6f255ad0f680
8908cfe140f5ea8a72645147ef183ff9703c7f1bc0f49456a8412e088f4caf0787944d7c54596836491bb6de7a87af6167f99c44729fe03cddaa6f255ad0f680
123

Lessons learned implementing AES in PHP using Mcrypt

The Advanced Encryption Standard (AES) is the successor of triple DES. When you need a standardized, secure, high performance symmetric cipher it seems like a good choice. Wi-Fi network traffic is encrypted with AES for instance. Also when you want to securely store data in a database or on disk you could choose AES. Many SSDs store data internally using AES encryption. PHP supports AES through “mcrypt”. On Debian based systems (like Ubuntu and Mint) you can install it using “sudo apt-get install php5-mcrypt”.

Rijndael-256 is not AES-256

In Mcrypt there is no cipher called “AES”, but Mcrypt supports “Rijndael”. Note that AES is based on the Rijndael cipher. So can we just use Rijndael for AES-128, AES-192 and AES-256? It seems straight forward since Mcrypt supports Rijndael-128, Rijndael-192 and Rijndael-256. But be aware of this pitfall:

AES-256 is different from RIJNDAEL-256. The 256 in AES refers to the key size, where the 256 in RIJNDAEL refers to block size. AES-256 is RIJNDAEL-128 when used with a 256 bit key. – source

So remember that only “Rijndael-128″ in “Cipher-block chaining” (CBC) mode is defined as the Advanced Encryption Standard (AES).

You can see this in effect in the following code:

<?php
echo "RIJNDAEL 128 CBC\n";
$max_key_size = mcrypt_get_key_size(MCRYPT_RIJNDAEL_128, MCRYPT_MODE_CBC)*8;
echo "Max key size: $max_key_size\n";
$block_size = mcrypt_get_block_size(MCRYPT_RIJNDAEL_128, MCRYPT_MODE_CBC)*8;
echo "Block size: $block_size\n";

This is the output:

RIJNDAEL 128 CBC
Max key size: 256
Block size: 128

Key padding woes

PHP is a very forgiving language. It allows you to define a variable containing a string and use it as an integer without a single warning. The Mcrypt functions handle different key sizes in the same forgiving way. Only 128, 192, and 256 bit keys are accepted in Rijndael-128, but if you feed Mcrypt a smaller key it will automatically pad it to an acceptable size using “zero” bytes. This is shown in the example below:

<?php
$text = "http://xkcd.com/1323/";
$enc = MCRYPT_RIJNDAEL_128;
$mode = MCRYPT_MODE_CBC;
$iv = mcrypt_create_iv(mcrypt_get_iv_size($enc, $mode), MCRYPT_DEV_URANDOM);
$keys = array();
// smaller than 128 bit key gets padded to 128 bit
$keys[] = pack('H*', "00112233445566778899aabbccddee");
$keys[] = pack('H*', "00112233445566778899aabbccddee00");
// larger than 128 bit key gets padded to 192 bit
$keys[] = pack('H*', "00112233445566778899aabbccddeeff00");
$keys[] = pack('H*', "00112233445566778899aabbccddeeff0000000000000000");
// larger than 192 bit key gets padded to 256 bit
$keys[] = pack('H*', "00112233445566778899aabbccddeeff001122334455667700");
$keys[] = pack('H*', "00112233445566778899aabbccddeeff00112233445566770000000000000000");
// larger than 256 bit key will be truncated (with a warning)
$keys[] = pack('H*', "00112233445566778899aabbccddeeff00112233445566778899aabbccddeeff00");
foreach ($keys as $key)
{ echo bin2hex(mcrypt_encrypt($enc, $key, $text, $mode, $iv))."\n";
}

The output of the above script:

a33d5ae5ca00e4523ea4e14baee2206351ad18b6728c95cafa56761f78d8f743
a33d5ae5ca00e4523ea4e14baee2206351ad18b6728c95cafa56761f78d8f743
6892e09f1ec5003cfdde3b2225fb3a094f0c17b8b12587d25fee1ece1b55146d
6892e09f1ec5003cfdde3b2225fb3a094f0c17b8b12587d25fee1ece1b55146d
794d046201f0dc775be3a6f5173024e07b7304c7e4313ab136333e7fc1df2831
794d046201f0dc775be3a6f5173024e07b7304c7e4313ab136333e7fc1df2831
PHP Warning:  mcrypt_encrypt(): Size of key is too large for this algorithm on line 19
685b12d5967f71ac4e881d228b5bcd7320a4c873f26fe25996229931c4566e92

Null padded strings

Because AES is a block cipher it cannot encrypt and decrypt a three byte plain text. Any plain text should be a multiple of the block size (which is 16). Again Mcrypt accepts any size of plain text and solves this “issue” for you by automatically padding it with zero bytes. Just like it does with the key. When decoding the plain text this may lead to some confusion. You can see this illustrated in the code below:

<?php
$enc = MCRYPT_RIJNDAEL_128;
$key = "secret";
$mode = MCRYPT_MODE_CBC;
$iv = mcrypt_create_iv(mcrypt_get_iv_size($enc, $mode), MCRYPT_DEV_URANDOM);
$text = "123";
$crypt = mcrypt_encrypt($enc, $key, $text, $mode, $iv);
$text = mcrypt_decrypt($enc, $key, $crypt, $mode, $iv);
var_dump($text);
var_dump(bin2hex($text));
var_dump(strtok($text,"\0"));
var_dump(bin2hex(strtok($text,"\0")));

That produces:

string(16) "123"
string(32) "31323300000000000000000000000000"
string(3) "123"
string(6) "313233"

In the above example the decrypted plain text seems to be padded with zero bytes. This is not true. It is actually the plain text, when fed to the cipher, that was padded with zero bytes. This makes the plain text after decrypting equal to the plain text before encrypting.

As you can see above a simple “strtok” call can remove everything from the first zero byte on from the string. NB: You should only do this if you are sure the original plain text does not contain any zero bytes. If it can contain zero bytes then make sure your plain text length is always a multiple of 16 and no padding occurs.

Ignored initialization vector

If the initialization vector does not have the right size (16 bytes), then Mcrypt does not pad it. Instead it outputs a warning and uses an empty initialization vector. This empty initialization vector is a series of 16 zero bytes. This behavior is shown in the code example below:

<?php
$enc = MCRYPT_RIJNDAEL_128;
$key = "secret";
$mode = MCRYPT_MODE_CBC;
$iv = "short_iv";
$text = "123";
$crypt = mcrypt_encrypt($enc, $key, $text, $mode, $iv);
$iv = pack('H*', "00000000000000000000000000000000");
$text = mcrypt_decrypt($enc, $key, $crypt, $mode, $iv);
var_dump($text);

The output of the above code is:

PHP Warning:  mcrypt_encrypt(): The IV parameter must be as long as the blocksize on line 7
string(16) "123"

So make sure you turn on warnings when using Mcrypt, it might hold some very valuable information.

Popular posts for “Count per Day” WordPress plugin

popularpostsLeaseWeb Labs runs on WordPress and we are using the “Count per Day” plugin to keep track of our visitors. One big advantage of Count per Day is that it does not use cookies, but counts the amount of unique IP addresses per day. These are reported as “visitors”, while the amount of page views are reported as “reads”. In our experience this is a very simple and reliable way of measuring the visitors on our site. Especially when you enable the option “Cache: Count visits with ajax” in the Settings, because that will prevent bots from showing up in the statistics.

Some posts are better and thus more popular than other posts. When people visit your site for a certain post, they may also be interested in other popular posts. This is why we wanted the most popular posts listed in the sidebar (as you can see on the left). Since the popularity of posts is changing all the time and we did not feel like maintaining a static list of posts in menu. Count per Day has a the shortcode “CPD_MOST_VISITED_POSTS” to generate this list for you. Unfortunately this code generates the list including numbers and a heading. This is not exactly the way we like it and thus we created a WordPress widget for our (and your) convenience.

The widget can be simply dragged into a widget area and allows you to:

  • Set the title for the list
  • Set the amount of days to take into account
  • Set the maximum number of items in the list
  • Show/hide the counts for the selected period
  • Show/hide the header that explains the counts

We contacted Tom Braider (the author of this fantastic plugin) and we hope he will include the widget in a next version. Until then you can install the widget by adding the following code to “widgets.php” using the theme editor:

if (isset($count_per_day)) {

// Creating the widget
class PopularPostsWidget extends WP_Widget {

  function __construct() {
    parent::__construct(
      // Base ID of your widget
      'cpd_popular_posts_widget',
      // Widget name will appear in UI
      'Popular Posts',
      // Widget description
      array( 'description' => 'Count-Per-Day Popular Posts'));
  }

  // Creating widget front-end
  // This is where the action happens
  public function widget( $args, $instance ) {
    $title = apply_filters( 'widget_title', $instance['title'] );
    $days  = $instance['days' ]+0;
    $limit = $instance['limit']+0;
    $head  = $instance['head' ]?true:false;
    $count = $instance['count']?true:false;
    // before and after widget arguments are defined by themes
    echo $args['before_widget'];
    if ( ! empty( $title ) ) {
      echo $args['before_title'] . $title . $args['after_title'];
    }
    // This is where you run the code and display the output
    global $count_per_day;
    $html = $count_per_day->getMostVisitedPosts($days, $limit, 1, 1, 1);
    if (!$head ) $html = preg_replace('/<small>[^<]*<\/small>/','',$html);
    if (!$count) $html = preg_replace('/<b>[^<]*<\/b>/','',$html);
    echo $html;
    echo $args['after_widget'];
  }

  // Widget Backend
  public function form( $instance ) {
    $title = isset($instance['title'])?$instance['title']:'Popular Posts';
    $days  = isset($instance['days' ])?$instance['days' ]:'7';
    $limit = isset($instance['limit'])?$instance['limit']:'10';
    $head  = isset($instance['head' ])?$instance['head' ]:false;
    $count = isset($instance['count'])?$instance['count']:false;
    // Widget admin form
    ?>
<p>
<label for="<?php echo $this->get_field_id( 'title' ); ?>"><?php _e( 'Title:' ); ?></label>
<input class="widefat" id="<?php echo $this->get_field_id( 'title' ); ?>" name="<?php echo $this->get_field_name( 'title' ); ?>" type="text" value="<?php echo esc_attr( $title ); ?>" />
</p>
<p>
<label for="<?php echo $this->get_field_id( 'days' ); ?>"><?php _e( 'Days:' ); ?></label>
<input class="widefat" id="<?php echo $this->get_field_id( 'days' ); ?>" name="<?php echo $this->get_field_name( 'days' ); ?>" type="text" value="<?php echo esc_attr( $days ); ?>" />
</p>
<p>
<label for="<?php echo $this->get_field_id( 'limit' ); ?>"><?php _e( 'Limit:' ); ?></label>
<input class="widefat" id="<?php echo $this->get_field_id( 'limit' ); ?>" name="<?php echo $this->get_field_name( 'limit' ); ?>" type="text" value="<?php echo esc_attr( $limit ); ?>" />
</p>
<p>
<label for="<?php echo $this->get_field_id( 'head' ); ?>"><?php _e( 'Show header:' ); ?></label>
<input class="widefat" id="<?php echo $this->get_field_id( 'head' ); ?>" name="<?php echo $this->get_field_name( 'head' ); ?>" type="checkbox" <?php echo $head?'checked="checked"':'' ?> />
</p>
<p>
<label for="<?php echo $this->get_field_id( 'count' ); ?>"><?php _e( 'Show counters:' ); ?></label>
<input class="widefat" id="<?php echo $this->get_field_id( 'count' ); ?>" name="<?php echo $this->get_field_name( 'count' ); ?>" type="checkbox" <?php echo $count?'checked="checked"':'' ?> />
</p>
    <?php
  }

  // Updating widget replacing old instances with new
  public function update( $new_instance, $old_instance ) {
    $instance = array();
    $instance['title'] = ( ! empty( $new_instance['title'] ) ) ? strip_tags( $new_instance['title'] ) : '';
    $instance['days' ] = ( ! empty( $new_instance['days' ] ) ) ? strip_tags( $new_instance['days' ] ) : '';
    $instance['limit'] = ( ! empty( $new_instance['limit'] ) ) ? strip_tags( $new_instance['limit'] ) : '';
    $instance['head' ] = ( ! empty( $new_instance['head' ] ) ) ? strip_tags( $new_instance['head' ] ) : '';
    $instance['count'] = ( ! empty( $new_instance['count'] ) ) ? strip_tags( $new_instance['count'] ) : '';
    return $instance;
  }
} // Class PopularPostsWidget ends here

// Register and load the widget
add_action( 'widgets_init', function (){ register_widget('PopularPostsWidget'); });

}