Latest Articles

Version 9.2 finished

Sunday, May 16, 2010 1 comment

It took quite some time, but finally I managed to finish the latest version. I have included some nice features, of which some may be interesting for my readers:
  • Highlighting for about 150 languages
  • Notifications for comments, both replies and follow-ups
  • Automatic twitter notifications about new blog posts
As usual, if you notice any bugs, let me know.

Start applications delayed after compiz

Saturday, May 1, 2010 0 comments

Since I have some applications which should start with gnome (which are Thunderbird, Empathy and X-Chat), I surely like to have them in popping up in certain places with fixed sizes. Since DevilsPie does not work very well with compiz, I'm using compiz' features itself to position the windows. The problem is, compiz needs some time to start, and thus applications may not be positioned at start. I solved this problem with a small bash script which is executed as startup program:
#!/bin/bash

function isCompizRunning()
{
    local result=$(dbus-send --print-reply \
                             --type=method_call \
                             --dest=org.freedesktop.compiz \
                               /org/freedesktop/compiz/dbus/screen0 \
                               org.freedesktop.compiz.list \
                   | wc -l)

    echo "$result"
}

while [ $(isCompizRunning) = 0 ]
do
    sleep 1
done

thunderbird &
empathy &
xchat &
Hopefully this will help some people having the same problem.

Raising private storage to the limits

Saturday, April 24, 2010 2 comments

A few months ago, I was looking for a new storage system, since my existing file server with it's 2.5TB of data were running full and I didn't want to put another drive in it. The existing file server was pretty loud and it made some trouble with the network connection since some time. I first looked into existing NAS systems, but with a good number of bays (between six and eight) were pretty expensive, while still suffering from the network bottleneck. They also were always just build with MDADM, a Linux software-RAID. My next idea was building a new, more silent file server, but with the right components I would also have gotten to a pretty high price, while still having the problem with the network bottlenecks.

Finally I made the decision that I, as I live alone, do not need a secondary machine anymore, which is running all the time. So I looked around for DAS systems, in the hope they would be cheaper. I came to the conclusion, that this was not the case, and thus looked for some enterprise models. Eventually I found my new storage, which should be a Sonnet Fusion DX800RAID with a hardware RAID controller and eight bays. It was pretty expensive with a price of 2200€, but that shouldn't be a show-stopper. The next thing would be to find the right drives for my new toy, and after getting some information from several friends, I decided to get eight Western Digital RE4-GP with 2 TB each. At that time, all decisions were made and it was time to order all the stuff for about 4400€.

After this month-long looking around, it should only take a week until everything arrived. When I came home at 18 o'clock I directly unpacked the new RAID system and started studying the installation manual. I was a bit shocked when I read that I first have to disassemble the trays from their transport bridges, but after an hour I had it done. It took me another 30 minutes then to prepare the drives with the threadlocker and mount them into their trays, but then I were ready to go on with the RAID itself.



The next part was inserting the RAID controller card into a PCI-E slot, which only took a couple of minutes. Time to tell you, I already bought a new mainboard a few weeks ago as I knew that I would need another PCI-E 8x slot. Anyway, after closing my PC again, I connected the DX800RAID with two mini-SAS cables to the PC and with a power cable to the power supply. Before I should boot the system, I surely had to insert all eight drives, which was done in a couple of minutes as well. Finally I should boot the system.

At this point, I didn't need any manual anymore, since it was straight forward. After the computer BIOS loaded, the BIOS of the RAID controller showed up and I pressed the hot-key to get into the setup for it. There I quickly setted up the RAID6 over all eight drives and rebootet the system. After Ubuntu loaded and the drivers and configuration tool were installed I had to see that the RAID had started rebuilding as it was degraded. As I didn't wanted to do anything wrong, I planned to wait until the rebuild was done. After playing QuakeLive for some hours, I returned to the system at 1 o'clock in the night.

The rebuild was done and I could start setting up an empty partition with DM-Crypt on it with an EXT4 partition, but it shouldn't be that easy as I thought. I were surely already telling some friends about my new toy, and one of them was going to tell me some things I had to consider when setting up everything. After talking a bit about the technical details of the RAID, we figured out that the chunk size was 128kb and the sector size was 521b, thus the empty partition had to start at 1536. This took a while, as GParted wasn't able to setup partition starts at sectors, and parted wasn't so easy to use. The nxt step should be to setup DM-Crypt correctly, and after a while we figured out that you can define the payload offset for encrypted data, which would be 1536 as well. Finally I could set up the EXT4 partition, and with the help of my friend we choosed the parameters for it wisely. At the time we were done, it was already 4 o'clock in the night, so I decided to get to bed after running a quick write-performance benchmark (which showed me nice 200 MB/s) and telling my friend that I owe him a beer.



When I came home the next day, I started copying all data from my old file server to the new RAID system with rsync, which should take a while. One day later, it wasn't even done with the first 1 TB disks (which were running in RAID1), so I decided to move it from the file server to my desktop PC, as I was going to use them as system disk for my next Ubuntu 10.04 installation anyway. This was a pretty hard task, as my huge graphics card blocked two HDD bays and I had to rearrange the disks a bit. I restarted the copying then from the local disk, which only took another 16 hours. The next day I started copying all other data parallely from my file server, which was done just 1.5 days later. Finally I could shut down my old file server forever.

My next step was to setup PAM_mount, as I didn't want to enter a password for DM-Crypt at boot time so I could boot the system remotely. Thanks to good tutorials on the internet, this was a pretty easy task. Eventually everything was done and I could get to bed with no loud machines running in my room anymore. Now it is saturday, one day after this multiple-day experience, and I have to leave now for a barbecue, so here is my conclusion:

It was a joy to setup everything, but less joy to copy all the data, but I learned a lot in this short amount of time. The performance of the DX800RAID is very good, and with the 9 TB of free space (12 TB in total plus 4 TB parity information) I should have a future-proof storage system for the next years. If anyone of you is looking for a good solution, I can recommend you the Sonnet Fusion products.

Zend Framework 2.0 is comming soon

Thursday, April 1, 2010 7 comments

Despite the fact that we wanted to wait for PHP 5.3 to be spreaded along all major Linux distributions, we decided to publish Zend Framework 2.0 in a shorter timeframe. This decision is supported by a clear demand by the users, who don't want to wait for PHP 5.3 support anymore. Since we won't be able to refactor all components we wanted, we are going to just convert everything to namespaces in ZF 2.0, and make the major refactoring in ZF 2.1. By this, you will get Zend Framework 2.0 in about two to three weeks.

By that time, we will also have moved to a new versioning system. As you know, we were valuating other versioning systems against the current SVN installation. Our first decision was, to move to GIT, but now we decided to go with Bazaar, which offers a much better infrastructure than GIT or SVN could do. We will surely keep a read-only SVN repository, so those of you who are still using svn:externals are not left behind.

Update - 2nd April 2010
This story was surely just an april fool ;)

Modern Application Design - Part 1

Friday, March 5, 2010 25 comments

Introduction
Based on the wishes by many people I was convinced to start a serial about modern application design. This serial will be completly based on my blog's source code, where I'll try to explain as many parts of my application as possible. As some parts were already covered in previous blog articles by me and other authors, I'll refer to them when the topics occur. In this first chapter I'm going to write about the basic setup of the application with some quirks which are rather not seen in most other setups.

Coding standard
This is probably the most important point, since it is leading through your entire application. There may still be people out there using some abitrary standard, if you can call it like that. But if there are more than only one person working on a project, you should definetly have one. For simplicity, I'm just using the Zend Framework coding standard.

Directory structure
As every project it always starts with choosing a specific directory structure. In my case I've choosen the Default modular directory structure proposed by Wil Sinclair, with some tiny modifications. One of those modifications is the additional library folder within my application folder, which holds the application library being non-specific to any module. I've also removed some folders of the proposal which I was not going to use. You may ask yourself, if a default directory structure is really important, and the answer is yes. When a developer comes from another Zend Framework project using the same structure, he will have no problem finding the way through your application within minutes.

Configuration
Many people nowadays use Zend_Tool to setup their initial project. As this automatically generates an INI config file, most developers stay with this kind of configuration. For many reasons I'm not doing that and instead using PHP config files, which I had written down about a year ago in another blog post.

Bootstrapping
Every new developer should be familiar with Zend_Application, which I and Matthew invented and published in Zend Framework 1.8. It has many benefits compared to the old and well-known procedural bootstrapping:
  • Your bootstraping code is portable and reusable
  • It is easier to test with PHPUnit
  • You can use single resource for individual scripts
  • It comes with many bootstrapping resources out-of-the-box
This is just a small list, but you should get the idea. I'll not go into detail about this component, as it is a pretty huge topic. Though you should know that I'm prefering _init* methods over resource plugins, since they are still partly portable and offer a greater performance bonus.

A few things to note about my quirks in bootstrapping: As I like to keep general services and models within the Default module, I am adding an additional resource autoloader for that module. I am also using module bootstrapping, as this automatically adds executes one bootstrap per module and registers a resource autoloader for it.

Routing
Many people just go with the default module route, which has some disadvantages. You have less control about the URIs and validation has always to be done in the controller. Also you will sooner or later get trouble with duplicate content, as many URIs can lead to the same controller-action. With custom routes, you can exacly define, how URLs should look like and do some rudimentary validation.

Exception handling
There are many kinds of exceptions, not the usual ones thrown by Zend Framework itself. For example, Zend Framework comes with three exceptions, which are "controller not found", "action not found" and, as of ZF 1.10, "route not found". Those can be used in the error controller to nicely display 404 (not found) pages. Additional to that, I'm using a custom App_PageNotFound exception, which is thrown by controllers if a request item could not be found. Another exception I use is the App_Permission exception. Those two will also be handled by the error controller to create appropriate output. All other exceptions will just generate a 500 (server error) page and log those exceptions.

To be continued …
In the next chapter of this serial I'm going to write about the service and model architecture used in my blog, stay tuned!