New design for LeaseWeb labs!

As you can see LeaseWeb labs has gotten a complete make-over. This is part of the two day hackaton at LeaseWeb that started today. As you may notice we also moved to HTTPS (SSL). Also we have a new theme that is based on Automattic’s twentyfifteen theme. This theme has lighter colors, a larger font size and is fully responsive (much better readable on smart phones). As you can see the LeaseWeb CDN banner has been moved to the bottom.

Design: old vs. new

For comparison, this is what the old design looked like:

old_layout

And this is the new design:

new_layout

I hope you like the changes. Anyway, let me know what your thoughts are using the comments, cheers!

 

Windows compatibility broken in Linux kernel (fixed)

linux_steam_broken

I’m running Ubuntu Linux and today I found that Steam did not work anymore (nor did some other Windows applications). Steam could not find my games and the store was not working. I remember installing some security updates. It turns out that Wine crashes on Ubuntu 14.04 LTS with the latest kernel update (3.13.0.59). There are several workarounds:

  1. Run “wineserver -p” in the terminal before starting Windows applications (like “steam”).
  2. Revert the kernel update with “sudo apt-get remove linux-image-3.13.0-59-generic” and “sudo update-grub”.
  3. Upgrade the kernel with “sudo apt-get install linux-image-generic-lts-vivid”.

Bug reports on the web

There are multiple places where people are discussing this bug:

I hope this helps you all… Happy gaming on Linux!

Update: A new kernel has been released, which fixes the problem: linux-image-3.13.0-61-generic (automatically installed)

 

Detecting torrent traffic on a Linux box

torrent_detection

At home I am sharing my Internet connection with several other family members. Sometimes my Internet is very slow with high latencies, causing my interactive SSH connections to stutter. The problem is always the same: somebody is downloading a torrent. And although I have no objection against torrent technology (it has many good applications), I hate it when I cannot work properly on my remote servers. So I decided to take action.

Wireshark and Tshark to the rescue

Wireshark has a command line version called “tshark”. It has a bittorent protocol analyzer and can be used to do Deep Packet Inspection (DPI). I decided to make a simple script that runs every 5 minutes and samples the network traffic for 10 seconds. After that it sends a report (top list, including packet count) of the local IP addresses that do the most torrent traffic (if there are any).  It can be ran using:

sudo tshark -a "duration:10" -Y bittorrent -f 'not port 80 and not port 22 and not port 443' | grep -o "192\.168\.1\.[0-9]\+" | sort | uniq -c | sort -rn | head | mail -E -s "LAN abusers" maurits@vdschee.nl

It is using postfix to send email via the gmail SMTP server (gmail account required). I am runnig the above in a cron job every 5 minutes. You may simply run this script on the gateway of your network. In case you can setup a port mirror on the switch of your up-link, then you can run this in promiscuous mode. Tshark will try to enable this mode by default, if it does not work, then check the FAQ here.

Blocking on detection

There are several ways to block the user that is abusing your network. I feel that temporary null routing the IP address is the simplest way. Additionally you may add an entry to your DHCP lease table to avoid that the user can simply request a new IP address. Filtering the good from the bad traffic is actually much more complicated. For one, because you need to find all the bad packets (as the software may try to avoid the block, switching protocols). If you really want to give it a try, you may look at netfilter string match. If you do, then make sure you enter good offsets and ranges to avoid negative performance impact on your network. Also I would not know where to get a maintained and complete set of protocol signatures.

torrent_utp_detection

Detecting uTP

If you are using the “Deluge” torrent client, you will be quickly detected by the above script. When you are using “Transmission” (another client) you may get away undetected. This is caused by the Micro Transport Protocol (aka “uTP”). This is a UDP based torrent protocol that cannot be recognized by Tshark yet. It is not very hard to actually make a custom rule that detects “uTP”. This is the custom filter:

sudo tshark -a "duration:10" -Y 'udp[8:5] == "\x64\x32\x3A\x69\x70" or bittorrent' -f 'not port 80 and not port 22 and not port 443' | grep -o "192\.168\.1\.[0-9]\+" | sort | uniq -c | sort -rn | head | mail -E -s "LAN abusers" maurits@vdschee.nl

The above command will detect also the “undetectable” uTP protocol. You may even extend the match a little as there are more fixed position bytes that can be matched.

Meta-programming: automate software creation

The secret to success in the business software field is to automate the software creation process. If you do so, you gain a big advantage as you have to program or customize less software than your competition. I have identified three types of meta-programming. They all have their advantages and disadvantages.

1) Higher subject abstraction

You can create software that sells “cars” that has, for instance, a table with “occasions”, containing columns for “brand” and “color”.

You can also make software that sells “products”, that have “properties”. Both could be tables, and there could be a foreign key between them. The properties table could contain the columns for “product” (a reference), “name”, “value” and “required”.

I see this pattern when I look at software that is not specialized for a niche. It is clear that generic web shop software like WooCommerce does not know what the web shop owner is going to sell and thus has no other choice than to generalize. It often also allows you to add “custom” fields, often with cryptic and generated names.

Is this bad? Yes and no. Yes, as this creates complex data structures that do not perform and are hard to understand. No, as you may have to do less software development. So in fact you pay your more effective software development by lower software performance and worse structured data.

2) Higher code abstraction

Programming language constructs like reflections, “generics” in C#, “method_missing” in Ruby, “magic methods” in PHP and “proxy classes” in Java enable programmers to write less code that is more generic or reusable.

Is this bad? Yes and no. Yes, when your code starts to get variables named “instance” and “class” it becomes hard to grasp what it is actually doing. No, as it may actually lead to less and more powerful code. In fact you are paying your more effective software development by less readable and harder to modify software.

The real problem is that this approach becomes really ugly when the software is aging. The code will start to contain lots of exceptions implemented as if/else statements in the wrong abstraction level.

3) Generating the code

When you generate code you can still add exceptions afterwards. Also you can still use specific, readable code and simple data structures for the domain you are automating. This actually gives you the best of both worlds: a ‘natural’ abstraction level for data and code, but still lower software development costs. Yes, it may give code duplication at first, but as soon as the code ages, you will see that the exceptions you will have to add actually justify this.

Another advantage of this method is that it can be applied in any programming language as no special language constructs are required.

Further reading

While you are at it, why not check out Rails 3 Generators: Scaffolding and MVC and Scaffolding for Rails Newbs? I’m sure you’ll love it!

BSOD during boot after disabling RAID in BIOS

The motherboard of my Acer M3920 has an on-board RAID controller which is part of the Intel Rapid Storage Technology. I added a 6TB WD Green drive, which was very easy using the easy-swap bay that the Acer M3920 has (see picture below). Unfortunately the RAID BIOS did not recognize the drive I inserted into the machine. The normal BIOS did recognize it, but in the RAID BIOS it was reported as a 1.4TB disk. Also Windows 7 Disk Management did not “see” the drive.

acer_m3920

Normally I would have updated the BIOS (as this would have likely solved the problem), but I could not find a newer BIOS on the Acer support site. So I decided to turn the RAID support off in the BIOS (switch the SATA mode from “RAID” to “AHCI”), since I did not used the RAID capabilities. This caused Windows to give a BSOD during boot with message “STOP: 0x0000007B” (see picture below). This actually is a cryptic way of Windows telling you “INACCESSIBLE BOOT DEVICE”.

bsod_inaccesible_boot_device

To avoid this error, the instructions in Microsoft KB article 922976  are not sufficient. They are sufficient when changing from AHCI to RAID, but not when changing from RAID to AHCI. Use the “regedit.exe” utility to change the “Start” key to value “0” for both of the following entries in your registry:

  • HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\msahci (as mentioned in the KB)
  • HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\atapi (not mentioned in the KB)

If you have this BSOD, reboot and set the SATA mode (in the BIOS) back to RAID. Then reboot and change the above registry settings. Finally change the SATA mode to “AHCI” and boot without BSOD and full access to your 6TB drive.

Special thanks go to M