Recent Comments
Archives
Visitors
  • 45375This month:
  • 504Today:
  • 19Currently online:



LeaseWeb CDN

PHP asset proxy increases website availability

remote_assets

Don’t you hate it when your site does not work, because you linked jQuery from “code.jquery.com” and that site is suffering connection problems? This may also happen with stylesheets or with font files. To counter this problem (but not lose the convenience of remote loaded assets) I created  an “asset proxy” in PHP. It will cache the assets in a cache folder on your web server, so that you do not have to worry about downtime of other services. You can configure how often the cache should be refreshed. When the external source is not available during a refresh the stale cache files will be used and there is no downtime at all!

proxy_assets

Install asset-proxy.php in your webroot. Then replace all references in your HTML from:

 href="http://fonts.googleapis.com/css?family=Droid+Sans:400,700"

to:

 href="/asset-proxy.php/fonts.googleapis.com/css?family=Droid+Sans:400,700"

Make sure you edit the list of allowed hostnames in the header of the PHP file and that you set an appropriate refresh time (in seconds). If the assets are not available upon refresh the stale files are served.

// hostnames for which "GET" requests can be proxied over "HTTP" (no ssl)
$hostnames = array(
	'fonts.gstatic.com',
	'maxcdn.bootstrapcdn.com',
	'netdna.bootstrapcdn.com',
	'fonts.googleapis.com',
	'ajax.googleapis.com',
);

// maximum age of a file before being refreshed
$refresh_age = 24*3600;

// directory where the cache resides (should exist and not be served)
$cache_dir = '/tmp/cache';

// strip the leading "/proxy.php/" from the URL
$url = substr($_SERVER['REQUEST_URI'], strlen($_SERVER['SCRIPT_NAME'].'/'));

// if there is no URL specified show bad request error
if(!$url || !strpos($url,'/')){
	header('Bad Request', true, 400);
	exit;
}

// get the hostname which should be the first segment (until the first slash)
$hostname = substr($url, 0, strpos($url, '/'));

// if the hostname is not in the list of allowed hostnames show forbidden error
if (!in_array($hostname, $hostnames)) {
	header('Forbidden', true, 403);
	exit;
}

// calculate the cached filename and check whether it already exists
$filename = $cache_dir.'/'.md5($url);
$file_exists = file_exists($filename);

// get the file age if the file exists
if ($file_exists) {
	$file_age = time()-filemtime($filename);
}

// if cache exists and is fresh, let's read the file, else retrieve it with cURL
if ($file_exists && $file_age<$refresh_age) {
	$result = file_get_contents($filename);
} else {
	// set some headers on the cURL call to pretend we are a user
	$sent_headers = array();
	foreach (array('User-Agent','Accept','Accept-Language','Referer') as $header) {
		$key = 'HTTP_'.strtoupper(str_replace('-','_',$header));
		if (isset($_SERVER[$key])) {
			$sent_headers[] = $header.': '.$_SERVER[$key];
		}
	}

	// make sure we do net get chunked, deflated or gzipped content
	$sent_headers[] = 'Accept-Encoding: ';
	$sent_headers[] = 'Cache-Control: max-age=0';
	$sent_headers[] = 'Connection: keep-alive';

	// initialize cURL with the URL, our headers and set headers retrieval on
	$curl = curl_init('http://'.$url);
	curl_setopt_array($curl, array(
			CURLOPT_HEADER => true,
			CURLOPT_RETURNTRANSFER => true,
			CURLOPT_BINARYTRANSFER => true,
			CURLOPT_HTTPHEADER => $sent_headers
	));

	// execute cURL call and get status code
	$result = curl_exec($curl);
	$status = curl_getinfo($curl, CURLINFO_HTTP_CODE);
	curl_close($curl);

	if ($status == 200) {
		// file was successfully retrieved
		if (file_put_contents($filename, $result)===false) {
			// show error on unsuccessful write
			header('Internal Server Error', true, 500);
			exit;
		}
	} else if ($file_exists) {
		// serve stale
		$result = file_get_contents($filename);
		// reset refresh timer
		touch($filename);
	}

}

// split the message in raw headers and body
if (strpos($result,"\r\n\r\n")!==false) {
	list($raw_headers,$body) = explode("\r\n\r\n", $result, 2);
} else {
	list($raw_headers,$body) = array($result,'');
}

// convert raw headers into an array
$raw_headers = explode("\n", $raw_headers);

// parse raw headers into received headers
$received_headers = array();
foreach ($raw_headers as $h) {
	$h = explode(':', $h, 2);
	if (isset($h[1])) {
		$received_headers[$h[0]] = trim($h[1]);
	}
}

// set certain headers for the output
$headers = array('Content-Type','Content-Encoding','Cache-Control','ETag','Last-Modified','Vary');
foreach ($headers as $header) {
	if (isset($received_headers[$header])) {
		header($header.': '.$received_headers[$header]);
	}
}

// replace the absolute URL's in the output
foreach ($hostnames as $hostname) {
	$body = preg_replace('/(https?:)?\/\/'.str_replace('.','\.',$hostname).'\//',
		$_SERVER['SCRIPT_NAME'].'/'.$hostname.'/', $body);
}

// set the new content length properly
header('Content-Length: '.strlen($body));

// echo the contents of the body
echo $body;

Best thing since sliced bread.. ;-) And only 128 lines of PHP code! Source code is on Github:

https://github.com/mevdschee/asset-proxy.php

 

AmsterdamPHP talks Docker at LeaseWeb

amsterdam_phpNext Thursday (December 18th) Robin Speekenbrink will be talking at LeaseWeb about Docker and virtualization in his talk titled “Meet the Phockers”.

Docker is an open platform for developers and sysadmins to build, ship, and run distributed applications. Consisting of Docker Engine, a portable, lightweight runtime and packaging tool, and Docker Hub, a cloud service for sharing applications and automating workflows, Docker enables apps to be quickly assembled from components and eliminates the friction between development, QA, and production environments. As a result, IT can ship faster and run the same app, unchanged, on laptops, data center VMs, and any cloud.- docker.com

At LeaseWeb we are very proud to be hosting this event. It was quickly “sold out” even though we had 100 seats available. One of the reasons the talk is so popular is that LeaseWeb has about 40 software developers that are all very interested in Docker technology themselves. So from yesterday another 25 seats are available. On the meetup site you can sign up and reserve a seat!

The schedule

19:00 - 19:30: Welcome Drinks 
19:30 - 20:30: Talk 
20:30 - 20:45: Raffle 
20:45 - 23:00: Social, drinks and Pizza

The location

You can easily find the location of LeaseWeb as it is near the IKEA in Amsterdam and it is situated in the corner of the A2 and A9 highways. To get directions and find the exact location you can use Google Maps. Or enter the following address in your satnav:

LeaseWeb Netherlands B.V.
Luttenbergweg 8
1101 EC Amsterdam
tel +31 20 316 288

Free as in free beer…

We have ordered lots of large pizza’s and we have juice, soda, draft beer and wine in our canteen. So please visit us, meet the nice people at LeaseWeb and enjoy our beautiful building and free parking.

Meetup link: http://www.meetup.com/AmsterdamPHP/events/168161902/

Web development in VB.net or C# on Linux with Nancy

nancy I will make my first Linux web application in Microsoft’s “.net” platform. Since I do not run Windows I am going to using Mono (“.net for Linux”) and the MonoDevelop IDE. I am not so much a fan of ASP.net, so I’ve chosen Nancy instead which is a light-weight and simple replacement of ASP.net.

Nancy is a lightweight web framework for the .Net platform, inspired by Sinatra. Nancy aims at delivering a low ceremony approach to building light, fast web applications.

Xamarin is the company that allows for cross-platform multi-target (iOS, Android, Mac & Windows) development in C#. They maintain a recent build of the MonoDevelop application (currently 5.5). This tutorial may either be used for programming in C# or VB.net on Linux. C# does have support for code completion, something VB.net is missing in MonoDevelop.

monodevelop

To install the Xamarin version we have to add the Xamarin key and the repository by executing the following commands in any Debian based Linux (like Ubuntu/Mint):

sudo apt-key adv --keyserver pgp.mit.edu --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
echo "deb http://download.mono-project.com/repo/debian wheezy main" | sudo tee /etc/apt/sources.list.d/mono-xamarin.list
sudo apt-get update

After that we can install everything using the following one-liner:

sudo apt-get install monodoc-base monodevelop mono-vbnc

Creating the project

After these installs (takes about 200 MB) you will have the “MonoDevelop” application (version 5.5) installed and you can start it from the “Development” menu. When you start it, it will ask you what you want to do. Choose to create an “Empty Project (VBNet)”. If you want to use C# then choose “Empty Project (C#)”.

error_project_dotnet_45

If you get an error stating “Error while trying to load the project”, “Project does not support framework ‘.NETFramework,Version=v4.5′”, then you may want to open the project XML file (either “test.vbproj” or “test.csproj”) and replace the value that is defined as the “TargetFrameworkVersion”:

<TargetFrameworkVersion>v4.5</TargetFrameworkVersion>

becomes:

<TargetFrameworkVersion>v4.0</TargetFrameworkVersion>

You can see below how that looks in my MonoDevelop:

traget_framework_version

And then we can go look for Nancy.Hosting.Self via Project > Add Packages

add_package_nancy

When the “Add packages” window opens type “Nancy.Hosting.Self” in the search field:

install_nancy

Select the package and click “Add Package” to add it to your project. Great! Now it is time for some code.

Nancy’s Hello World example

First we make sure we start our host. In the Application.vb file.

Option Explicit On
Option Strict On
 
Imports System

Public Class Application

  Public Shared Sub Main()
    Dim host As Nancy.Hosting.Self.NancyHost
    host = New Nancy.Hosting.Self.NancyHost(New Uri("http://localhost:8080"))
    host.Start()
    Console.ReadLine()
  End Sub

End Class

And then we create a new Class “HelloModule” and add this code to it.

Option Explicit On
Option Strict On

Imports System

Public Class HelloModule
    Inherits Nancy.NancyModule

    Public Sub New()
        Me.Get("/") = AddressOf Hello
    End Sub

    Public Function Hello(ByVal parameters as Object) As String
        Return "Hello world"
    End Function

End Class

If we use C# then the Application.cs file looks like this:

using System;

public class Application
{
    static void Main(string[] args)
    {
        Nancy.Hosting.Self.NancyHost host;
        host = new Nancy.Hosting.Self.NancyHost(new Uri("http://localhost:8080"));
        host.Start();
        Console.ReadLine();
    }
}

And we create a second class “HelloModule” with the following content.

using System;

public class HelloModule: Nancy.NancyModule
{
    public HelloModule ()
    {
        Get["/"] = hello;
    }

    public String hello(Object parameters)
    {
        return "Hello world";
    }
}

Running your Nancy application

If you press run and you might get the following error:

Error CS0246: The type or namespace name `Uri' could not be found.
Are you missing an assembly reference?

Click on “Project > Edit References”, then type “System” in the search field, select the reference and click “OK”. If you get the following error:

/usr/lib/mono/4.5/Microsoft.VisualBasic.targets: Error: Error executing task Vbc:
Argument cannot be null. Parameter name: pathToTool

In case you get the above mentioned error, again, go to “Build\General” and under the section “Build Engine” uncheck the “Use MSBuild build engine …”

no_msbuild

And now open a browser and go to: http://localhost:8080/

If the page does not load it maybe because the application already exited. This can be solved by clicking “Run on external console” under “Project > Options > Run > General”:

run_on_ext_console

And now press run, everything should work and you should see some “Hello World” magic!

Finally..

You will learn that the MonoDevelop environment works flawlessly for C#. It also works quite good for VB.net, but it has some rough edges. The Nancy framework will feel very familiar if you have experience with frameworks like Sinatra, Silex or Flask. Mono is great for programmers that prefer Linux and in most application development it does not limit your possibilities. If you are going to run Nancy code in a production environment, then you may want to read about running Nancy behind a Nginx webserver.

NB: If you like this article you may also want to look into running ASP.NET vNext on Linux as well explained by Graeme Christie.

10 reasons why PHP is better than Python

“There are only two kinds of languages: the ones people complain about and the ones nobody uses” – Bjarne Stroustrup,

People wonder: Did he really say that? Yes, he did. If you wonder who Bjarne Stroustrup is: He created the C++ programming language. Many people believe that Python is “pretty cool” and PHP is “really bad”. But as we all know, there is truth in the saying:

It’s a poor carpenter who blames his tools.

I believe both good and bad software can be written in any language. And I should probably also quote Joel Spolsky who calls “language wars” a “fruitless debate”. But nevertheless, if you program in PHP and run into one of these Python “evangelists”, then the following list may come in handy.

10 reasons why PHP is better than Python

  1. Python hosting, hard to find and expensive, while cheap PHP hosting is everywhere.
  2. Python cannot be mixed with HTML (needs a template library), while PHP can.
  3. Python has no proper encapsulation (private keyword), while PHP has.
  4. Python is hardly used in the real world, while something as big as Facebook is built on PHP.
  5. Python has a great community, but it is not comparable to PHP’s.
  6. Python has some books and tutorials, but PHP has way more of them.
  7. Python does not have the live docs (famous forum-like reference manual) like PHP has.
  8. Python does not have a steep learning curve, but PHP is still easier to learn.
  9. Python indentation for code blocks is prone to errors, while PHP uses curly braces.
  10. Python lexical scoping is a mess (‘global’ and ‘nonlocal’ keywords fix this), while PHP behaves as expected.

To wrap up

PHP has come a long way. Today it is a mature language that executes fairly speedy. Agreed that it has some quirky naming of it’s built-in functions, but hey.. that’s the price you pay for backwards compatibility.

Hint: Make sure to also check out this page on Python.org that is a good reference when comparing Python to PHP.

 

Tutorial: Apache 2.4 as reverse proxy

This post explains how to configure Apache 2.4 (the version that comes with Ubuntu 14.04) as a fully transparent reverse proxy. If you have a single website that has multiple paths that are actually run by different web applications then this tutorial may be for you.

reverse_proxy

The proxy will serve both web applications from their own virtual host configuration. These may be on the same machine as shown below using the loop-back addresses 127.0.0.1 and 127.0.0.2 or on different machines if you use their (internal) IP addresses.

Site: http://www.yourwebsite.com/
App1: http://www.yourwebsite.com/app1 = http://127.0.0.1/app1
App2: http://www.yourwebsite.com/app2 = http://127.0.0.2/app2

This is the directory structure in which I want to load the various web apps:

maurits@nuc:/var/www/html$ ll
total 28
drwxr-xr-x 4 root root  4096 Dec  1 21:43 ./
drwxr-xr-x 3 root root  4096 Apr 21  2014 ../
-rw-r--r-- 1 root root 11510 Apr 21  2014 index.html
drwxr-xr-x 2 root root  4096 Dec  1 21:45 app1/
drwxr-xr-x 2 root root  4096 Dec  1 21:45 app2/

In this tutorial we run the web applications on the same paths as on the proxy. This means that the web apps run in a subdirectory, even on the machines behind the proxy. This avoids the need of rewriting and thus keeps this setup simple and easy to debug.

Setting up the reverse proxy in Apache 2.4

What we are going to do is setup a reverse proxy. First we load the “proxy_http” module in Apache 2.4 using:

sudo a2enmod proxy_http
sudo service apache2 restart

Let’s setup the reverse proxy virtual host configuration in “/etc/apache2/sites-available/yourwebsite-proxy.conf” like this:

<VirtualHost *:80>
ServerName www.yourwebsite.com
DocumentRoot /var/www/html
ProxyPreserveHost On
ProxyPass /app1 http://127.0.0.1/app1
ProxyPass /app2 http://127.0.0.2/app2
</VirtualHost>

The virtual host configuration of app1 in “/etc/apache2/sites-available/yourwebsite-app1.conf” looks like this:

<VirtualHost 127.0.0.1:80>
ServerName www.yourwebsite.com
DocumentRoot /var/www/html
...
</VirtualHost>

And the virtual host configuration of app2 in “/etc/apache2/sites-available/yourwebsite-app2.conf” looks like this:

<VirtualHost 127.0.0.2:80>
ServerName www.yourwebsite.com
DocumentRoot /var/www/html
...
</VirtualHost>

Lets enable all sites and reload Apache using:

sudo a2ensite yourwebsite-proxy yourwebsite-app1 yourwebsite-app2
sudo service apache2 reload

Note that this works as the virtual host configurations with a specified IP address will be matched first. The “ProxyPreserveHost” will make sure the “Host” header in the request is not rewritten. The lack of a “ProxyPassReverse” will make sure that there is no rewriting done on the response.

Showing the correct remote IP address

It is important to understand that in the above setup, the proxied web application will only see a different “REMOTE_ADDR” environment variable, since there is absolutely no rewriting going on. The real visitor address is passed along in “X-Forwarded-For” header. This is a comma separated list and the last entry holds the real client IP address.

If you are on Apache 2.4, like in Ubuntu 14.04, you can correct the reported remote address by loading the “remoteip” module like this:

sudo a2enmod remoteip
sudo service apache2 restart

Add the “RemoteIPHeader” and “RemoteIPInternalProxy” directives to the virtual host configurations:

<VirtualHost 127.0.0.1:80>
ServerName www.yourwebsite.com
DocumentRoot /var/www/html
RemoteIPHeader X-Forwarded-For
RemoteIPInternalProxy 127.0.0.0/8
...
</VirtualHost>

Note that the “RemoteIPInternalProxy” you must specify the internal IP address of the proxy. To test if you did it right you can run a PHP script that calls “phpinfo()”. If you see that the “REMOTE_ADDR” value is not set to the proxy, then it is working.

Adding headers to the upstream request

We want to make Apache2 add upstream headers and therefor we need to load the “headers” module in Apache 2.4 using:

sudo a2enmod headers
sudo service apache2 restart

Next, we have to adjust the reverse proxy virtual host configuration in “/etc/apache2/sites-available/yourwebsite-proxy.conf” like this:

<VirtualHost *:80>
ServerName www.yourwebsite.com
DocumentRoot /var/www/html
ProxyPreserveHost On
RewriteEngine On
RequestHeader add X-SSL off
RewriteRule ^/app1/(.*) http://127.0.0.1/app1/$1 [P,L]
RewriteRule ^/app2/(.*) http://127.0.0.2/app2/$1 [P,L]
</VirtualHost>

In this example we add a “X-SSL” header with the value “off” to the proxied request. If you want to add headers to the response you can use the “Header” directive.

If you have any questions, please use the comments below.