Archives
Visitors
  • 13533This month:
  • 2998Today:
  • 35Currently online:



LeaseWeb CDN

Become a better programmer in 30 minutes

Programmers nowadays should all know these 3 very important rules:

  1. Don’t use floating points for money
  2. Store date/time in “UTC” timezone
  3. Always use the “UTF8″ character set

Tom Scott brilliantly explains the above 3 topics in videos that roughly last 10 minutes each. He explains them clearly, with depth and in a passionate way. They are listed below. Start the video’s by clicking on them.

1. Don’t use floating points for money

…never ever…

utf8_video

2. Store date/time in “UTC” timezone

…yes, always…

timezone_video

3. Always use the “UTF8″ character set

…no exceptions…

floating_points_video

IMHO every programmer should watch these videos, because they are very educational. And “Yes”, it will takes you 30 valuable minutes of your precious time. But “No”, you will not regret it, because they teach you some valuable and elementary lessons. Lessons that apply on any software project, written in any programming language.

Thank you Tom Scott, you rock! Keep doing videos on your YouTube channel!

Memcache Bundle for Symfony updated!

memcache_v2

We have released version 2 of our Symfony Memcache Bundle. The 3 biggest improvements are:

  • Added Windows support
  • PHP 5.4+ without notices and/or warnings
  • High availability support with redundancy configuration

This actually was a major refactoring. We switched from the PECL “memcached” to the PECL “memcache” (without ‘d’) extension. Only version 3 (which is still in beta) of this extension is supported currently, but this version is quite stable actually. Most of the features of the bundle have not been altered. The Memcache(Pool) object is deliberately not abstracted. This is done to allow for unrestricted access to the underlying implementation. The flags parameter is now supported and this is the third argument on a “set” instruction. You may have to alter your code for this, which may be annoying, but we promise to keep the API stable in this major version that may again last for a few years.

Some articles we have written about the bundle are:

Let us know your experience with this new version.

Download LswMemcacheBundle from Github

Limit concurrent PHP requests using Memcache

When you run a website you may want to use nginx reverse proxy to cache some of your static assets and also to limit the amount of connections per client IP to each of your applications. Some good modules for nginx are:

Many people are not running a webfarm, but they still want to protect themselves against scrapers and hackers that may slow the website (or even make it unavailable). The following script allows you to protect your PHP application from too many concurrent connections per IP address. You need to have Memcache installed and you need to be running a PHP web application that uses a front controller.

Installing Memcache for PHP

Run the following command to install Memcache for PHP on a Debian based Linux machine (e.g. Ubuntu):

sudo apt-get install php5-memcache memcached

This is easy. You can flush your Memcache data by running:

telnet 0 11211
flush_all

You may have to restart apache for the Memcache extension to become active.

sudo service apache2 restart

Modifying your front controller

It is as simple as opening up your “index.php” or “app.php” (Symfony) and then pasting in the following code in the top of the file:

<?php
function firewall($concurrency,$spinLock,$interval,$cachePrefix,$reverseProxy)
{
  $start = microtime(true);
  if ($reverseProxy && isset($_SERVER['HTTP_X_FORWARDED_FOR'])) {
    $ip = array_pop(explode(',',$_SERVER['HTTP_X_FORWARDED_FOR']));
  }
  else {
    $ip = $_SERVER['REMOTE_ADDR'];
  }
  $memcache=new Memcache();
  $memcache->connect('127.0.0.1', 11211);
  $key=$cachePrefix.'_'.$ip;
  $memcache->add($key,0,false,$interval);
  register_shutdown_function(function() use ($memcache,$key){ $memcache->decrement($key); });
  while ($memcache->increment($key)>$concurrency) {
    $memcache->decrement($key);
    if (!$spinLock || microtime(true)-$start>$interval) {
      http_response_code(429);
      die('429: Too Many Requests');
    }
    usleep($spinLock*1000000);
  }
}
firewall(10,0.15,300,'fw_concurrency_',false);

Add these lines if you want to test the script in stand-alone mode:

session_start();
session_write_close();
usleep(3000000);

With the default setting you can protect a small WordPress blog as it limits your visitors to do 10 concurrent(!) requests per IP address. Note that this is a lot more than 10 visitors per IP address. A normal visitor does not do concurrent requests to PHP as your browser tends to send only one request at a time. Even multiple users may not do concurrent requests (if you are lucky). In case concurrent requests do happen they will be delayed for “x” times 150 ms until the concurrency level (from that specific IP) is below 10. Other IP addresses are not affected/slowed down.

If you use a reverse proxy you can configure this (to get the correct IP address from the “X-Forwarded-For” header). Also if you set “$spinLock” to “false” then you will serve “429: Too Many Requests” if there are too many concurrent requests instead of stalling the connection.

This functionality is included as the “Firewall” feature of the new MindaPHP framework and also as the firewall functionality in the LeaseWeb Memcache Bundle for Symfony. Let me know what you think about it using the comments below.

Fast dynamic DNS with cron, PHP and DuckDNS

ducky_icon_mediumMy home connection has a 200 mbit cable Internet connection with 20 mbit up. Great for running a server, but every two days my ISP changes my IP address. When this happens I cannot connect to my home network anymore using VPN. Annoying, but certainly a (programming) challenge to me. The simple solution for this is to use a dynamic DNS solution. The name DynDNS popped up in my head, but apparently they are not free anymore (bummer). That’s why I chose to use the free dynamic DNS service “DuckDNS“. Then I realized that I do want a fast update of my dynamic DNS entry when my IP address changes, but I do not want to hammer DuckDNS. That’s why I wrote a small script to achieve this. You find it below.

DuckDNS PHP script to avoid hammering

On my website I installed the following PHP script that will call DuckDNS if the IP address of the caller has changed. It is must be called with a post request that holds a shared secret. This will avoid bots (or hackers) to change the DNS entry. Note that additionally HTTPS (SSL) is used to guarantee confidentiality.

<?php
// settings
$domains = 'cable-at-home'; // cable-at-home.duckdns.org
$token = 'eb1183a2-153b-11e5-b60b-1697f925ec7b';
$ip = $_SERVER['REMOTE_ADDR'];
$file = '/tmp/duckdns.txt';
$secret = 'VeryHardToGuess';
// compare secret
if (!isset($_POST['secret']) || $_POST['secret']!=$secret) { http_response_code(403); die(); }
// compare with current ip
if ($ip==file_get_contents($file)) { http_response_code(304); die('OK'); }
// create url
$url = "https://www.duckdns.org/update?domains=$domains&token=$token&ip=$ip";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$result = curl_exec($ch);
curl_close($ch);
// if success update current ip
if ($result!='OK') { http_response_code(400); die($result); }
file_put_contents($file,$ip);
die('OK');

Install this script somewhere in your Apache “DocumentRoot” and name it “duckdns.php”.

Cron script that runs every minute

I installed the following cron job on my server that runs in my home and is connected with cable to the Internet using the “crontab -e” command:

* * * * * /usr/bin/curl -X POST -d 'secret=VeryHardToGuess' https://somedomain.com/duckdns.php

Every minute this cron job executes a curl call to the duckdns.php PHP script on my website (somedomain.com). Only if the IP address is changed the call to DuckDNS (https://www.duckdns.org/update) is made to update the IP address. This avoids hammering the DuckDNS service, but also allows you to get the fastest response to an IP address change.

Installation

Note that in order to make this work you have to create an account at DuckDNS and then modify the “$domains” and “$token” parameters in the PHP script accordingly. You need to change “somedomain.com” in the cron job with the URL of your website. Also do not forget to replace “VeryHardToGuess” in both the PHP script as the cron job with a real secret. Any questions? Use the comments below!

When (not why) “Agile” and especially Scrum are terrible

This article is a response to “Why ‘Agile’ and especially Scrum are terrible” as posted on Michael O. Church’s blog. He is not so much attacking the agile manifesto, as he he is attacking the reality of “agile”, the way it portraits itself and the excesses.

If you read the comments on Reddit then you find that there are two camps: scrum advocates and scrum haters. The haters complain how scrum leads to bad behavior and the advocates argue that this behavior is caused by doing scrum wrong. In this post I will focus on this behavior that, both parties agree, should be avoided.

1) Being judged personally

You should not be judged on hour by hour productivity as a software developer, because constant daily competition is not healthy. What particular tasks, technical debt or research tasks each developer worked on in a sprint is irrelevant. Peer pressure is the most effective way to keep people productive and looking at individual performance kills peer pressure. If there is micro-management going on, then that has nothing to do with being “Agile”.

Although there may be a high level of transparency, that does not mean that management should be evaluating people’s performance at the end of each sprint. The story points are designed to prevent management from doing this. The retrospective allows for criticizing the performance and effectiveness of people in the team by the team.

Especially web-based “Agile” registration tools must be used with care (or avoided). They are not part of “scrum” as white-boards and stickies are preferred. Also they may add lots of bureaucracy and may be misused to draw conclusions. Conclusion that may destroy the valuable trust relationship between employer and employee as they are more likely to be wrong than right.

If you see “agile” and “scrum” as tools to achieve “accountability” or “predictability” you are missing the point. It is merely a method to talk about work done instead of work to be done.

2) Developers being considered the lowest rank

The “product owners” and “scrum masters” do NOT outrank “team members”. Although they prioritize the user stories, they should not not choose which ones get executed. Also they should not assign tickets. If they feel superior or they are treated (or paid) as such, then something is wrong and not in the last place because good team members are all-round technical experts and these are extremely hard to find.

Team members are stakeholders as well. They can not only choose which tickets to work on (from the top of the stack), but also add stories (often technical debt) and argue why they are important.

One of the goals of agile is to eliminate blame and fear in the workplace. In a good team people do not fear to say “I was stuck with my code all day and got nothing done” during a stand-up. People should understand their pain and instead of blaming them offer some help.

3) Short term focus

Under Agile, technical debt can pile up not being addressed, because non-technical business people decide on priorities. They will not see a problem until it’s far too late or, at least, too expensive to fix it.

This pileup may be the result of  a straightforward application of “scrum/agile” principles. The focus may easily shift to only things that bring business value or “visible features” as I like to call them. These principles might work when imposed/implemented by an engineering team onto itself, but they break apart instantly the moment the management tries to impose them from outside.

There’s no reason why major refactoring can’t be tackled as a story. And having metrics to measure this (e.g. steadily increasing time to deliver features) should make it easier, not harder, to make the business case.

Conclusion

Short term work under high scrutiny is not fun nor agile, avoid it.