Symfony2 Memcache session locking

In one of the previous posts we wrote about session reliability. Today we will talk about “locking session data”. This is another session reliability topic and we will look at the problems that may occur in Symfony2 and how to solve them.

Session locking

Session locking is when the web server thread acquires an exclusive lock on the session data to avoid concurrent access. Browsers use HTTP 1.1 keep-alive and would normally just use one open TCP connection and reuse that to get all dynamic content. When loading images (and other static content) the browser may decide to use multiple TCP connections (concurrent) to get the data as fast as possible. This also happens when using AJAX. This may (and will most likely) lead to different workers (threads) on the web server answering these concurrent requests concurrently.

Each of the requests may read the session data update and write it back. The last write wins, so some writes may get lost. This can be countered by applying session locking. The session lock will prevent race conditions from occurring and prevent any corrupted data appearing in the session. This can easily be understood by looking at the following two images.

session-access-without-lockingsession-access-with-locking

The left image shows concurrent requests without session locking and the right shows concurrent requests with session locking. This is very well described in this post by Andy Bakun. Note that the above images are also from that post. Reading the Andy Bakun post allows you to truly understand the session locking problem (and the performance problems that AJAX may cause).

Symfony2 sessions

In Symfony2 one would normally use the NativeFileSessionHandler, which will just use the default PHP session handler. This works flawless in most cases. PHP uses “flock” to acquire an exclusive lock on the local filesystem. But when you scale out and run a server farm with multiple web servers you cannot use the local filesystem. You might be using a shared (NFS) filesystem and run into problems with “flock” (see the Linux NFS FAQs). If you use the database you may run into performance problems, escpecially when applying locking.

This leaves Memcache or Redis as options for session storage. These are fast key/value stores that can be used for session storage. Unfortunately the Symfony2 session storage implementations for Memcache (in Symfony) and Redis (in phpredis) do not implement session locking. This potentially leads to problems, especially when relying on AJAX calls as explained above. Note that other frameworks (like CakePHP) also do not implement session locking when using Memcache as session storage. Edit: This post has inspired the guys from SncRedisBundle and this Symfony2 bundle now supports session locking, which is totally awesome!

Custom save handlers

One can write “Custom Save Handlers” as described by the Symfony2 documentation:

Custom handlers are those which completely replace PHP’s built in session save handlers by providing six callback functions which PHP calls internally at various points in the session workflow. Symfony2 HttpFoundation provides some by default and these can easily serve as examples if you wish to write your own. — Symfony2 documentation

But you should be careful, since the examples do not implement session locking.

LswMemcacheBundle to the rescue

At LeaseWeb we love (to use) Memcache. Therefore, we have built session locking into our LswMemcacheBundle. It actually implements acquiring a “spin lock” with the timeout set to PHP’s “max_execution_time” (defaults to 30 seconds). The spin lock tries to acquire the lock every 150 ms (configurable). It will also hold the lock for a maximum time of the PHP “max_execution_time”. By using Memcache’s built-in key expire mechanism, we can ensure the lock is not held indefinitely.

This (spin-lock) implementation is a port of the session locking code from the memcached PECL module (written in C). Our bundle enables locking by default. If you want, you can disable the locking by setting the “locking” configuration parameter to “false” as described in the documentation.

This session locking code was also ported to SncRedisBundle and submitted as PR #109. LswMemcacheBundle is open-source and can be found on our GitHub account:

https://github.com/LeaseWeb/LswMemcacheBundle

6 Responses to “Symfony2 Memcache session locking”

  • Vlad:

    This is a solution to a problem that should not exist in the first place. Even if the application is not RESTful, there are certain principles in REST that imho should be applied to any web application because they solve a set of base problems that should never exist and spawn more problems like this one.

    Resource/method access should be idempotent (even for C in CRUD — I know a lot of users that “instinctively” double click on buttons) and talking about browsers opening parallel connections (per user/session) should be treated like an exception. Even if your app handles eg. on-page images in a way that requires some kind of session update, the app design should be such that does not require session locking (except for data atomicity which should really be solved by the session storage layer).

    Premature optimization aside, any app should really be written from the start being agnostic about the number of servers it is run on, and keeping in mind that session locking is access serialization.

  • Maurits van der Schee (Innovation Engineer):

    @Vlad: Interesting take on the matter. You use the word “should” a lot. This tells me you might be having real-world experience with this problem. Is that true? IMHO session locking should be done on a per-variable basis as argued by Andy Bakun back in 2006 (see: http://thwartedefforts.org/2006/11/11/race-conditions-with-ajax-and-php-sessions/#why-per-variable-locking). Unfortunately 7 years later none of the major frameworks has implemented this (yet).

  • Vlad:

    Indeed, not so much with PHP as I primarly develop in Python now. I do maintain a legacy PHP app which precisely does what I mentioned – on-page image handling with the app itself (ACL checks). With PHP’s default file handled sessions, they’re locked automatically, so even if you don’t do any updates it’s gonna serialize access (per user) which is a visible performance impairment.

    So if your page has to load 100 images (a 10 x 10 matrix if thumbnails) they’re all gonna load one by one, serially unless you do custom session management with “smart” locking (on write only, atomically, by the storage layer eg. db). Add to that round trip times and application overhead, the user experience deteriorates.

    Modern frameworks tend to deal with a lot of things automatically – including JS aggregation/compression, CSS preprocessing, which is cached yes, but requires app access (ie. not a static resource in context of a web server) so this is all gonna be serialized unless locked smartly. I’ve seen a Drupal installation with a custom template and a lot of CSS and JS files do 200 requests just to render a single page.

    Since Python does not deal with sessions automatically like PHP, I’ve learned of many methods to do session management – something you never think about in PHP until you really need it. One of those is cookie-based session management which does not use anything on the server side – everything is stored and signed in the cookie and if used minimally (eg. just as an authenticated user identifier) things are a lot easier to do and much more scalable.

    Also, with AJAX, interactivity can be handled asychronously rendering the “flash” messages (that require sessions) obsolete. There is no more POST – 302 – GET cycle with forms if you use async forms, so sessions really lose on the significance.

    So it all boils down to proper app design. REST dislikes sessions for a reason – even if technically one can consider sessions as yet another RESTful resource and treat it like that.

  • Maurits van der Schee (Innovation Engineer):

    @Vlad: Point taken. Another apporach would be to just accept that every user has one TCP connection for dynamic requests and several others for static requests. This is the simple reality with traditional web pages that don’t use AJAX. By just accepting this reality one can build great performing and scalable websites and web applications. This model is much simpler and complexity has its price too.

  • Vlad:

    @Maurits

    That’s just one of the approaches that work in a specific set of situations where there is clean separation of purely static and purely dynamic content. Though I don’t agree with putting AJAX in an exclusive relation with “traditional web pages” that assume “one TCP connection”. One can still use AJAX in “one TCP connection” context, eg. contact forms.

    BTW, “one TCP connection” is perhaps broad generalization, but a wrong one. Keep-alive renders it wrong ;) Perhaps you meant “one request-response cycle at a time”.

  • Maurits van der Schee (Innovation Engineer):

    @Vlad: I really appreciate your take on this. It helps me (and hopefully also other readers) to fully understand the matter. I do have to comment on one thing. I said using no AJAX was using “one TCP connection”. You say using “one TCP connection” one can still use AJAX. Both are true IMHO.

Leave a Reply