PyCon US 2014 is being held in Montreal and the conference was last weekend (April 11-13).
PyCon is the largest annual gathering for the community using and developing the open-source Python programming language.
For all of you that did no go, the full conference is available as 138 YouTube videos:
- Python Epiphanies
- Opening Statements – PyCon 2014 (2014/04/11)
- mrjob: Snakes on a Hadoop
- Mining Social Web APIs with IPython Notebook
- Lightning talks – Saturday morning
- How to formulate a (science) problem and analyze it using Python code
- Hands-on intro to Python for beginning programmers
- Getting Started with Django, a crash course
- Exploring Machine Learning with Scikit-learn
- Diving deeper into Machine Learning with Scikit-learn
- Descriptors and Metaclasses – Understanding and Using Python’s More Advanced Features
- 0 to 00111100 with web2py
- What Is Async, How Does It Work, And When Should I Use It?
- Twisted Mixing
- Turn Your Computer Into a Server
- The Young Coder: Let’s Learn Python
- The State of Crypto in Python
- The Python Pipeline: Why you should reach out to local teachers
- Teaching Python: To Infinity and Beyond
- TDD for web applications, from scratch
- Straightening Out AngularJS with Python
- So you want to be a full-stack developer? How to build a full-stack python
- Shiny, Let’s Be Bad Guys: Exploiting and Mitigating the Top 10
- Search 101: An Introduction to Information Retrieval
- Realtime predictive analytics using scikit-learn & RabbitMQ
- Python Scraping Showdown: A performance and accuracy review
- Python + Geographic Data = BFFs
- Python for Social Scientists
- Python 3/2 Web Development with Pyramid
- Puppet Modules: Apps for Ops
- PostgreSQL Proficiency for Python People
- Pickles are for Delis, not Software
- Lightning talks – Sunday morning
- Let’s Learn Twisted Python
- Kneel And Disconnect: Getting The Fastest Connection Out Of A Hostname
- Keynote – John Perry Barlow
- IPython in depth: high productivity interactive and parallel python
- Introduction to Web (and data!) Scraping with Python
- Introduction to SQLAlchemy
- Introduction to Regular Expressions
- Introduction to game programming
- Improving automated testing with py.test
- Import-ant Decisions
- How to Get Started with Machine Learning
- Hello Physical World: A Crash Course on the Internet of Things
- Hands-on with Pydata: how to build a minimal
- Getting Started with SaltStack
- Getting Hy on Python: How to implement a Lisp front-end to Python
- Generators: The Final Frontier
- For Lack of a Better Name(server): DNS Explained.
- Flask by Example
- Faster Python Programs through Optimization
- Fan-in and Fan-out: The crucial components of concurrency
- Enough Machine Learning to Make Hacker News Readable Again
- Dynamics and Control with Python
- Django for Web Designers and Front End Developers
- Distributed task processing using Celery
- Distributed Computing Is Hard, Lets Go Shopping
- Decorators: A Powerful Weapon in your Python Arsenal
- Data Wrangling for Kaggle Data Science Competitions — An etude
- Contribute with me! Getting started with open source development
- Computer science fundamentals for self-taught programmers
- Character encoding and Unicode in Python
- Castle Anthrax: Dungeon Generation Techniques
- Cache me if you can: memcached, caching patterns and best practices
- By Your Bootstraps: Porting Your Application to Python3
- Build your own PiDoorbell! – Learn Home Automation with Python
- Building and breaking a Python sandbox
- Blending art, technology, and light, Python for interactive and real time
- Beyond Defaults: Creating Polished Visualizations Using Matplotlib
- Bayesian statistics made simple
- Application Deployment State of the Onion
- Ansible – Python-Powered Radically Simple IT Automation
- An Introduction to Twisted
- All Your Ducks In A Row: Data Structures in the Standard Library and Beyond
- Advanced methods for creating decorators
- Writing RESTful web services with Flask
- Which messaging layer should you use if you want to build
- What is coming in Python packaging
- Upgrade your Web Development Toolchain
- Unit Testing Makes Your Code Better
- Track memory leaks in Python
- The Sorry State of SSL
- The Day of the EXE Is Upon Us
- Technical on-boarding, training, and mentoring.
- Subprocess to FFI: Memory, Performance, and Why You Shouldn’t Shell Out
- So You Want to Build an API?
- Software Engineering Research for Hackers: Bridging the Two Solitudes
- Software Carpentry: Lessons Learned
- Smart Dumpster: Employing Python to Report Real-Time Resource
- Set your code free: releasing and maintaining an open-source Python project
- See Docs Run. Run, Docs, Run!
- Sane schema migrations with Alembic and SQLAlchemy
- REST is not enough: Using Push Notifications to better support your mobile clients
- Quick Wins for Better Website Security
- Python packaging simplified, for end users, app developers
- Python in the Browser: Intro to Brython
- Pushing Python: Building a High Throughput, Low Latency System
- Programming an Autonomous 20 Foot Blimp with Python
- PostgreSQL is Web Scale (Really)
- Postgres Performance for Humans
- Performance Testing and Profiling: A Virtuous Cycle
- Outreach Program for Women: Lessons in Collaboration
- My big gay adventure. Making, releasing and selling an indie game made in python.
- Multi-factor Authentication – Possession Factors
- Localization Revisited
- Lightning talks – Sunday morning
- Lightning talks – Saturday evening
- Know Thy Neighbor: Scikit and the K-Nearest Neighbor Algorithm
- Keynote – Van Lindberg
- Keynote – Jessica McKellar
- Keynote – Guido Van Rossum
- Keynote – Fernando Perez
- It’s Dangerous to Go Alone: Battling the Invisible Monsters in Tech
- Introduction to SQLAlchemy Core
- Introduction to Docker
- In Depth PDB
- Hitchhikers Guide to Participating in Open Source
- Getting Started Testing
- Garbage Collection in Python
- Games for Science: Creating interactive psychology experiments
- Fast Python, Slow Python
- Farewell and Welcome Home: Python in Two Genders
- Django: The good parts
- Diving into Open Data with IPython Notebook & Pandas
- Discovering Python
- Developing Flask Extensions
- Designing Poetic APIs
- Designing Django’s Migrations
- Deliver Your Software In An Envelope
- Data intensive biology in the cloud: instrumenting ALL the things
- Closing address – PyCon 2014 (2014/04/13)
- Cheap Helicopters In My Living Room
- Building the App
- A Scenic Drive through the Django Request-Response Cycle
- Analyzing Rap Lyrics with Python
- Advanced techniques for Web functional testing
- 2D/3D graphics with Python on mobile platforms
Now you need some serious time off to view all that!
“The Expert” is the title of the short comedy sketch Lauris Beinerts (Google+) posted a last week on his Youtube account. It went viral and is heading for half a million views in under seven days. It is about a “Funny business meeting illustrating how hard it is for an engineer to fit into the corporate world.” The video stars Orion Lee, James Marlowe, Abdiel LeRoy, Ewa Wojcik, and Tatjana Sendzimir, and is written & directed by Lauris Beinerts. It is based on a short story by Alexey Berezin titled “The Meeting” (in Russian) or translated by Google.
“I can do absolutely everything, I’m an expert!”
At LeaseWeb we understand…
At LeaseWeb we understand how hard it is for an expert to operate in the corporate world. So, if you have been in too many meetings like the one so well portrayed above, then you may want to take a look at our career offerings for engineers. Currently we are looking for:
- Senior Engineer Virtualization and Storage
- Hosting Engineer Virtualisation and Storage
- Hosting Engineer Cloud
- Software Engineer CDN
- Software Engineer Cloud
- Infrastructure Security Engineer
- Infrastructure Engineer Cloud
We employ over 360 people in the Netherlands, Germany, the UK, and the US. Despite the company size, we try to keep things informal around the offices, with few layers of management. The members of the OCOM Group (LeaseWeb, EvoSwitch, FiberRing, DataXenter) hail from over 34 different nationalities, making English the language of choice.
2048 is hot! It may be considered the next Flappy Bird. I implemented 2048 in the C programming language, so you can run it on your Linux server on the console (in text mode). You can find the 2048.c project on Github.
You can move the tiles in four directions using the arrow keys: up, down, left, and right. All numbers on the board will slide into that direction until they hit the wall and if they bump into each other then two numbers will be combined into one if they have the same value. Each number will only be combined once per move. Every move a new number 2 or 4 appears. If you have a 2048 on the board you have won, but you lose once the board is full and you cannot make a move.
I chose to store the field in a two-dimensional array called board that is 4×4.
   press   up key   
Since I want the board to be addressed as board[x][y], the board consists of a set of four columns size four. So the first column is the array [2,0,4,0]. If we press the up button this should become [2,4,0,0]. The function “slideArray()” is reponsible for this. This function will “slide” the numbers in the arrays like this:
[2,0,4,0] => [2,4,0,0] [0,4,0,8] => [4,8,0,0] [8,0,2,2] => [8,4,0,0] [2,2,2,2] => [4,4,0,0]
The algorithm can just use board[x] to point to a column and work directly on that. In pseudocode, this is what the algorithm does:
- walk over the array from the first to the last number - for each original number in the array that is not zero - look backwards for a target position that does not contain a zero (unless it is position zero) - if the target position does not contain the original number use the next position - if the target position is different from the original position - add the number to the number on the target position - replace the original number by zero
The above algorithm executed will do all transformations, it will:
[2,2,2,2] => [4,4,0,0] [0,8,2,2] => [8,4,0,0]
But there is a problem, it will also do this:
[2,2,4,4] => [8,4,0,0]
The two’s are merged into a four and then the first four is merged into that making it an eight. This is wrong. It should be doing this:
[2,2,4,4] => [4,8,0,0]
This is avoided by adding a “stop” variable that will be initially set to zero, but when a merge has been done it will be set to the merge position plus one. This will make sure any next slide will stop before it merges into this number again, since double merges are not allowed.
To prevent complex programming I use a function rotateBoard that rotates the board 90 degrees counter-clockwise. This allows the moveLeft(board) to be implemented as:
rotateBoard(board); moveUp(board); rotateBoard(board); rotateBoard(board); rotateBoard(board);
As long as you rotate four times in total everything works as expected. This method is not very efficient, but reduces the complexity of the code.
Compiling and running
Since 2048.c is a single C file it is easy to get running, just execute the following commands:
wget https://raw.githubusercontent.com/mevdschee/2048.c/master/2048.c gcc -o 2048 2048.c ./2048
This will run on most machines. If not, then execute the following command to install the compiler:
sudo apt-get install build-essential
The LswGuzzleBundle adds Guzzle API call functionality to your Symfony2 application. It is easy to use from the code and is aimed to provide full debugging capabilities. source: Github
On Packagist, you can see that in the past two years we have published 10 LeaseWeb Symfony2 bundles. The latest and tenth addition to the list is the LswGuzzleBundle. It is an alternative for the LswApiCallerBundle, so it also is a bundle that helps you to call API’s using cURL. The difference is that this bundle uses Guzzle as a wrapper for cURL.
Guzzle profiler and debugger
This bundle also adds a debug panel to your Symfony2, where you can find the cURL API calls Guzzle makes in your application. It provides profiling and extensive debugging capabilities to your Symfony2 application. We feel it is an essential bundle if you create an application that does not work with a database, but instead has an API as data store backend. Guzzle allows you to specify the parameters and and available calls in a Guzzle service description. Since this allows the bundle to understand the structure of the call, it will display the parsed parameters and output. This is an unique feature of this bundle.
The bundle is MIT licensed. You can find it at:
As always there is a readme documentation on GitHub to get you started.
We have set a goal to create a flexible and extendable automated testing framework, which should expand test coverage for as many LeaseWeb applications functionalities as possible. Tests should be also run on CI (Jenkins in our case). We are writing our tests on Java with help of several tools:
- Webdriver is a driver that contains programming interface for controlling all kinds of possible actions in browser.
- TestNG is a testing framework. It structures, groups, and launches tests. It also generates testing reports.
- Maven is a software project management and comprehension tool. It manages all dependencies and different flows for building a project.
1. We created a Maven project with next structure:
src/main/java – contains packages with Page Objects. Each package represents separate application and can contain packages with general methods and objects of the framework.
src/main/resources – here we store config files and other resources that are used in our tests.
src/test/java – contains packages with tests. Each package contains tests related only to one application or a test flow that goes through the couple of applications.
This structure is important for a project compiling in Maven.
2. The main goal was to write tests in a readable manner where each step is one single line of code that clearly describes what it does, and at the same time hides all actions that need to be done by Webdriver “under the hood” in our Page Object classes. By this we make it really easy to read and understand what our test does for every team member – from developers to product managers and even sales people.
3. Test run. We decided to use TestNG for several reasons:
a. People are moving from JUnit to TestNG once they start to write integration and end-to-end tests (e.g. with Selenium). On unit tests level you can put up with JUnit deficiencies, but once you start to write integration and end-to-end tests, JUnit stands in your way and TestNG comes to help you out.
b. No need to create your own Thread objects! You can use @Test annotation with ThreadPoolSize, and invocationCount parameters to run parallel tests.
c. Parameterization. A good testing framework should make it simple, and avoid copy\paste coding. Alas, the default JUnit implementation of the parametrized tests is unusable. In short, JUnit expects all test methods to be parameter-less. Because of this constraint, parameterized tests with JUnit looks awkward because parameters are passed by constructor and kept as fields. People who work with JUnit usually test their code with only one set of parameters. I suspect this is because parameterized tests are unusable with JUnit. And this is bad because such tests are really useful. TestNG allows you to use @DataProvider annotation, which allows you to parameterize your tests however you like and improve your test coverage dramatically with less lines of code in your tests.
d. Parameters to passing the tests:
- Configure your tests so they, for example, invoke GUI methods on different servers/ports?
- Run tests using MySQL database instead of h2?
- Run GUI tests with Opera, Firefox, Chrome (or even IE 6)?
- It is all easy to do with @Parameters annotation in tests.
e. Groups of tests. Each test class (and method) can belong to multiple groups, and with TestNG it’s very easy to run only selected group with @Test (group) annotation. You can easily set in CI which group of tests you want to run – smoke, GUI, API, etc.
f. Test dependencies. It’s not desirable practice with unit tests, but when it comes to integration tests, pretty often you cannot avoid it. This is where TestNG becomes very helpful and easy to use.
g. Reporting. TestNG provides more detailed and helpful reports in comparison with JUnit.
All these settings and tweaks can be specified in textng.xml file, which is used by TestNG to run tests both on your local dev machine and on CI Server. It will declare which tests should be run or skipped and contains information about groups, packages, and classes. This file is located in the root project folder.
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" > <suite name="CAS" verbose="1" > <test name="RegressionLogin" > <classes> <class name="OrderFlowTests"> <methods> <include name="orderFlow" /> </methods> </class> <class name="LoginOppTests" > <methods> <include name="successLoginToOpp" /> </methods> </class> </classes> </test> </suite>
4. Our pom.xml setup.
The beauty of Maven for us, is that it keeps and maintains all dependencies that we use in our project. It makes it much easier to setup the project on a new machine, and also makes a lot easier to run the project on CI. We need at least two plugins to start maven-compiler-plugin и maven-surefire-plugin (in the tags of which we have written our testing.xml). We made few tweaks in the plugin configuration section to show our testing.xml file, and also specify which file naming formats should be considered as tests for TestNG. File is located in to the root project folder.
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>NewLeaseWebSite</groupId> <artifactId>NewLeaseWebSite</artifactId> <version>0.0.1-SNAPSHOT</version> <dependencies> <dependency> <groupId>org.testng</groupId> <artifactId>testng</artifactId> <version>6.3.1</version> </dependency> <dependency> <groupId>org.seleniumhq.selenium</groupId> <artifactId>selenium-firefox-driver</artifactId> <version>2.37.1</version> </dependency> <dependency> <groupId>org.seleniumhq.selenium</groupId> <artifactId>selenium-java</artifactId> <version>2.26.0</version> </dependency> </dependencies> <build> <testSourceDirectory>src/tests/java</testSourceDirectory> <resources> <resource> <directory>src/main/resources</directory> <excludes> <exclude>**/*.java</exclude> </excludes> </resource> </resources> <pluginManagement> <plugins> <plugin> <artifactId>maven-compiler-plugin</artifactId> <version>3.1</version> <configuration> <source>1.7</source> <target>1.7</target> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.16</version> <configuration> <suiteXmlFiles> <suiteXmlFile>testng.xml</suiteXmlFile> </suiteXmlFiles> <includes> <include>**/Test*.java</include> <include>**/*Tests*.java</include> <include>**/*Tests.java</include> <include>**/*Test.java</include> <include>**/*TestCase.java</include> </includes> </configuration> </plugin> </plugins> </pluginManagement> </build> </project>
As a result, we have a test framework that can be easily extended, should not require much effort for maintenance in the future, and can be run on any CI server with several simple setup steps.