Thomas Koch - PHP

Free Software in companies

nospam@example.com (Thomas Koch) — Fri, 06 Aug 2010 10:46:25 +0200

At work we currently have a discussion: I'd like to develop some components as Free Software. My bosses like the idea, but the client goes nuts only thinking about it.
So as part of the discussion I thought to collect those companies who actively advertise their Free Software. I know about the big ones, but it would be even more interesting to get a list of succesful small companies that share at least part of their inhouse projects. On the other hand it would not be too interesting to list full Free Software companies like Red Hat.
It would also be fine to share this list in some wiki (FSFE?) when it grows larger. For some companies I also list very popular projects from my point of view.

big companies

Adobe
Facebook: Cassandra, Thrift, Scribe, HipHop for PHP, ...
Yahoo: YUI, Design Pattern Library, YSlow, Hadoop, ZooKeeper
Twitter
Google: Chromium Browser, Google Summer of Code, Android, Google Web Toolkit (GWT), Protocol Buffers - data interchange format, Java Collections Library, unladen-swallow - faster implementation of Python, ...
IBM: Developerworks (Tutorials), Eclipse, International Components for Unicode (ICU)

middle sized companies

Digg

small companies

101tec (now Datameer): Katta (distributed lucene on Hadoop), zkClient

Update: Need to add Danga (Gearman), Liip.ch with Okapi, Flux CMS, Jackalope, rackspace

NoSQL summer at lake constance

nospam@example.com (Thomas Koch) — Thu, 17 Jun 2010 09:31:00 +0200

Have you already started to try out these new storage/database things commonly referred to as NoSQL? (For you google-bot, I'm talking about CouchDB, Cassandra, HBase, Hadoop, Hypertable, MongoDB, Tokyo Cabinet, etc). Well, it's one thing to install and use them but another to understand all the computer science background about them.
Therefor many developers around the world thought to make this summer a NoSQL summer. Local meetings are held in many cities: London, Los Angeles, New York, Paris, lake constance ...
If you like to meet for a beer and learn and discuss about some of the hottest stuff in computing, then come around! Please select the dates when you can join and the papers you'd like to discuss. You can subscribe to email announcements or ical and rss feeds at the lake constance nosql summer page.

Zookeeper for web developers

nospam@example.com (Thomas Koch) — Fri, 19 Feb 2010 10:42:48 +0100

Have you ever developed any kind of distributed system? When doing so for the first time, you're very likely to fall in the trap of the Fallacies of Distributed Computing. I've done so, you'll do so too.
Now zookeeper is an application, that helps you implement many distributed protocols on top of it. The hard work of implementing fault tolerance, assuring consistency and that kind of stuff is done by zookeeper in the background. A zookeeper cluster consists of at least three servers running zookeeper (zk). A client can connect to each zk server and issue read and write requests. Zookeeper guaranties that a write either fails or is consistent and that a read will get you the most recent state.
Zookeeper exposes a filesystem like hierarchy of so called znodes. Every znode can have children but also stores data. The data stored in a znode is assumed to be small (less then 1MB). Clients can subscribe to different events on a znode and will be notified of changes in the znode itself of it's children.
Some usage examples come to my mind, which could be especially interesting for PHP developers:

save the PHP session in zk and have it therefor available to all web servers
save a shopping card
save the online status of a user (for a chat system)
synchronize configuration files
master election: many monitoring servers, but only one is active at any given time
logging system: bookkeeper is a zk contribution, that receives your logs and assures they'll be kept save
create a lock on a document in a CMS

Companies that use zk include Facebook, Yahoo, Rackspace and as rumors tell also twitter.
Now this is all kind and sweet, the only sad thing is, there's no PHP binding yet. There are bindings for Java, C, Python and Perl. So if you're desperately searching what PHP extension you should write next, just take the zk C binding and expose it to PHP!
Maybe you're a student and would like to participate in the Google Summer of Code? It should be possible to find a mentor for this project either in the PHP or zk project.
For GSOC the project could include a PHP session implementation in the extension code and the possibility of persistent zk sessions across PHP calls.
If you're in the region of eastern Switzerland: There'll be a presentation about zookeeper at the Webtreff Kreuzlingen on one of the next mondays.

Going to FOSDEM 2010

nospam@example.com (Thomas Koch) — Sat, 30 Jan 2010 20:52:08 +0100

I'm going to fosdem 2010.

These are things I'd like to attend:

time	room	topic	speaker
10:45-11:30	Janson	Promoting Open Source Methods at a Large Company	Brooks Davis
13:15-13:45	AY	Packaging Java Software for Debian	Thomas Koch
13:40-13:55	Ferrer	CiviCRM: Common goals of FOSS and Not For Profit Organisations	Xavier DUTOIT
14:00-14:45	H.1308	The Maemo Community Council: a case-study in governance	Dave Neary
14:00-15:30	Guillissen	LPI exam session 1	Klaus Behrla
14:00-14:45	Janson	What is my system doing - Full System Observability with SystemTap	Mark Wielaard
14:30-15:00	H.2213	Building Federated Social Networks on XMPP	Tuomas Koski, Simon Tennant
15:00-15:45	Janson	Ganglia: 10 years of monitoring clusters and grids	Bernard Li
15:00-15:30	H.2213	XMPP and the Social Web	Alard Weisscher, Laurent Eschenauer
15:30-16:00	H.2213	PubSub Gone Wild: Info Sharing at Mediamatic	Ralph Meijer
16:00-17:30	Guillissen	LPI exam session 2	Klaus Behrla
16:00-16:15	Ferrer	SIP Communicator: Skype-like conf calls with SIP Communicator	Emil Ivov
16:20-16:35	Ferrer	Kamailio (OpenSER) 3.0.0: redefinition of SIP server	Daniel-Constantin Mierla
16:40-16:55	Ferrer	asterisk: An introduction to Asterisk Development	Mark Michelson
17:00-17:15	Ferrer	csync: Roaming Home Directories	Andreas Schneider
17:30-18:00	H.2213	Jingle Nodes: An Open Alternative to Skype	Tiago Camargo
18:00-18:30	H.2213	Multi-User Jingle: Voice and Video Conferencing with XMPP	Dafydd Harries, Sjoerd Simons
18:15-19:00	H.1302	Spacewalk: Linux Systems Lifecycle Management	Marcus Moeller, Sandro Mathys


09:15-10:00	H.1302	Linux distribution for the cloud	Peter Eisentraut
10:00-10:45	Janson	RepRap - Manufacturing for the Masses	Adrian Bowyer
10:30-12:00	Guillissen	LPI exam session 3	Klaus Behrla
11:00-11:45	AW1.120	My life with HBase	Lars George
11:00-11:45	Janson	Tor: Building, Growing, and Extending Online Anonymity	Andrew Lewman
12:15-13:00	H.1302	Cross distro packaging with (top)git	Thomas Koch
12:15-13:00	H.1308	Shared libraries in Debian	Sune Vuorela
13:00-14:30	Guillissen	LPI exam session 4	Klaus Behrla
13:15-14:00	AW1.120	CouchDB, a database designed for the web and more	Benoit Chesneau
14:00-14:45	Janson	Large scale data analysis made easy - Apache Hadoop	Isabel Drost
15:00-16:30	Guillissen	LPI exam session 5	Klaus Behrla
15:30-16:15	H.1308	Continuous Packaging with Project-Builder.org	Bruno Cornec
15:30-16:15	AW1.120	Comparing the MapReduce way in CouchDB with the SQL way in a RDBMS	Stéphane Combaudon
16:15-16:45	AW1.120	Designing a scalable content management system on NoSQL technologies	Evert Arckens
16:15-17:00	H.1308	Debian Secrets: power tools for power users	Wouter Verhelst

Exploring Eclipse, PDT and Eclim

nospam@example.com (Thomas Koch) — Sat, 24 Oct 2009 16:48:24 +0200

Back in 2006 I learned, that KWrite (a simple text editor) was not enough to write PHP code with. I tried eclipse, but it seemed too complex and was way too slow on my old laptop.

So inspired by the Comfortable PHP editing with VIM series of Tobias Schlitt I learned VIM.

My blogpost about Vim as an IDE for PHP is a popular page and I often get requests about details of my setup. Please don't ask me anymore, since I intend to move away from this setup!

Some weeks ago I've been at a conference (GearConf) about team collaboration. Some of the tools presented there (like Mylyn) made me jealous of the java people.

So it was time to reconsider Eclipse. However editing text with what's called an "editor" in eclipse is a pain to say the least. But life's good and somebody already created eclim. Eclim allows you to keep using gvim for text editing but integrates with Eclipse to give you code completion, manual lookup, code projects and more.
This alone may not convince you, since I already have this integrated in vim. But there are at least three major advantages of vim+eclipse:

There's not only the integration for PHP but also Java, Javascript, Python, Ruby, C/C++, CSS, HTML and XML
Eclipse comes with a bunch of other cool plugins (like Mylin)
The code completion from eclipse is much more powerful then anything you can do with VIM. For example it also proposes you the correct classmembers after foo:: or $bar->

My collegue Jiayong just installed the bundled version of PDT with eclipse, ran the eclim installer and was ready.

However I'm much more addicted to pain, so I took the Ubuntu packages of eclipse 3.5.1, linked the pdt runtime download and all dependencies in a separate directory, checked out the GIT repository of eclim, configured the classpath in the ant buildfile and build eclim from source. The deployment of eclim can be looked up from the ant buildfile.
There are some more tweaks to do, which I can point out later, if you should dare to follow this road. You shouldn't have a need for this until you'd be interested to become famous and beloved and package PDT or eclim for Debian.

Do you already have experiences with eclim+pdt? Are there tutorials I've not found? Are you interested to come with me to explore this setup?

eZ Components Template is not slow...

nospam@example.com (Thomas Koch) — Fri, 09 Oct 2009 13:40:42 +0200

... you may just not have used it appropriately. Fabien Potencier has blogged about templating engines, compared their speed and finally concluded that his own templating engine is the fastest. Not only that, but my beloved eZ Components Template engine, which we use with great pleasure at YMC, came out as the slowest in his benchmark. (Althought he complimented it as "probably the one which have the most features".)

Dear Fabien, I think it's not fair to blog a benchmark without publishing the code you used to do the benchmark! But even without the code and without doing my own benchmark, I can offer some explanations, why eZ Components may have been the slowest in your benchmark and why you've compared apples with oranges. I take the compiled template code from the Twig site:

CODE:

/* Hello {{ name }} */ class __TwigTemplate_1121b6f109fe93ebe8c6e22e3712bceb extends Twig_Template { public function display($context) { $this->env->initRuntime(); // line 1 echo "Hello "; echo (isset($context['name']) ? $context['name'] : null); } }

Thinks to keep in mind from the Twig compiled template:

There's direct output via echo (which is captured by Twig and not directly sent out)
The template variables are sent in the array $context
There's no error check, whether the required template variable 'name' is actually sent to the template
The content of the variables is not escaped

Now let's compare this with the compiled code produced by eZ Components Template. First the template:

CODE:

{use $name} {use $items} Hello {$name} {foreach $items as $item} * {$item} {/foreach}

And the resulting code:

CODE:

<?php // Generated PHP file from template code. // If you modify this file your changes will be lost when it is regenerated. $this->checkRequirements(1,array("disableCache" => false)); $i_output = ""; if ( !isset($this->send->name)) { throw new ezcTemplateRuntimeException( sprintf("The external (use) variable '%s' is not set in template: %s and called from %s", 'name', $this->template->stream, ( sizeof($this->template->streamStack) >= 2 ? $this->template->streamStack[sizeof($this->template->streamStack) - 2] : 'the application code') ) ); } if ( !isset($this->send->items)) { throw new ezcTemplateRuntimeException( sprintf("The external (use) variable '%s' is not set in template: %s and called from %s", 'items', $this->template->stream, ( sizeof($this->template->streamStack) >= 2 ? $this->template->streamStack[sizeof($this->template->streamStack) - 2] : 'the application code') ) ); } $i_output .= " \nHello "; $i_output .= htmlspecialchars($this->send->name); $i_output .= "\n \n"; foreach ($this->send->items as $t_item) { $i_output .= "* "; $i_output .= htmlspecialchars($t_item); $i_output .= "\n"; } return $i_output; ?>

The differences are:

The output is collected in a variable, not with PHP's output buffering. I don't know whether this has a performance impact, but in any case it allows for much greater flexibility in the implementation of the template engine. Thus it's possible for example to
The template variables do not come in a simple array but as an object of type ezcTemplateVariableCollection. This is surely slower as an array, but also allows to do great things.
My favorite eZC Template feature: Every variable needs to be declared at the top of the template! I wish PHP would be that strict.
And now the point which may be responsible for the observed performance impact: eZC Template by default escapes all variables with htmlspecialchars. This is of course configurable and so I could compare template execution time with escaping turned on and of: For an array of thousand elements it more then doubles the time.

So in conclusion: Please post the benchmark code, so that we can debug it and please compare the template engines with the same features turned on or off!

Timeline of PHP quality assurance tools

nospam@example.com (Thomas Koch) — Sat, 26 Sep 2009 16:58:46 +0200

I'll be giving a talk on PHP quality assurance tools next week at the gearconf in Düsseldorf. The event is focused on team collaboration and most of the audience will come from other languages then PHP. Since I'm not a contributor to any QA tool I'll center around the user perspective and the process of establishing these tools in a small to middle web company like YMC.
As an outline for the first part I thought to give an historical view on QA in PHP and therefor made a timeline of PHP QA milestones:
The dates as text:

~1997 JUnit
6.4.2002 PHPUnit 0.1
28.8.2003 PHPUnit 1.0.0-alpha1 Release
13.7.2004 PHP5.0.0
14.7.2004 release PHPUnit2
30.1.2006 eZ Components 1.0
~sept 2006 ZF uses PHPUnit (issue #374)
4.10.2006 first integration of Selenium in PHPUnit
1.7.2007 Zend Framework 1.0
18.11.2007 first release of PHPUnderControl
19.11.2007 Project Mess Detection in PHPUnit
6.2.2008 first release PHP_Depend
march 2009 qualityassuranceinphpprojects.com, thePHP.cc

main sources:

Did I get all this right? Are there things I should have added?

PHP SAPI to run gearman workers

nospam@example.com (Thomas Koch) — Mon, 20 Jul 2009 09:17:44 +0200

Gearman is a system to run jobs on multiple servers. We at YMC are working know to integrate gearman in our cluster solution. Since we're a PHP shop, the jobs should be written in PHP.
There already is a solution with the gearman PHP extension by James M. Luedke. But I wanted to run multiple workers on one machine and having them use an opcode cache. The solution is a dedicated PHP binary (SAPI), which runs many worker threads in one process. The code (see /sapi/gearman_worker) in branch gearman_worker is already usable and I'd be happy to receive comments on it. After all I'm not (yet) a C expert.

TODOS:

graceful stop on SIGTERM
include the possibility to run cron-like scripts in a loop
pause, if available memory is too low
documentation
get it into upstream

eZ Components: The Good The Bad and the Ugly

nospam@example.com (Thomas Koch) — Wed, 17 Jun 2009 11:07:21 +0200

I've been nominated again for the eZ Components Award. Thanks to eZ Systems for honour and to me employer YMC for the food!
So this is a good occasion to assemble a list I've had in mind for a longer time. In a big project like eZ Components, which has been programmed by many contributors over a long time it is natural, that there are some better and some worse parts. So which components are the good, the bad and the ugly? Maybe this kind of overview could also be done by other users of other libraries too. Then we could assemble a big comparison table of framework or library components and one could pick the best parts of each.
You'll notice that even if I consider myself a power user of eZ Components, I won't give a rating for every component, because there are still a few, I've not intensively used by now. So let's complete this table in a wiki somewhere together!

So IMHO:

Component	rating	justification
Authentication	works	the component should better not handle $_SESSION itself
Cache	not used yet
Configuration	not used yet
ConsoleTools	great
Database	great
DatabaseSchema	bad	You want to have more control over your database schema.
Document	great	first ReST parser in PHP? Also many other formats!
EventLog	great
Feed	great
Graph	great
ImageAnalysis	not used yet
ImageConversion	not used yet
Mail	great
MvcTools	works	too few documentation of best practices
PersistentObject	great
Search	ugly	sorry, but it's not flexible enough for more then the most basic needs
SignalSlot	bad	haven't seen any use case that convinces me
SystemInformation	bad	doesn't comply to ezc coding guidelines
Template	great	stable, fast, easily extensible
Translation	works	Misses the important XLIFF format, but can be extended
Tree	not used yet
Url	bad
UserInput	ugly	see my attempt for a new form component
Webdav	not used yet	but looks very promissing!
Workflow	great

The numbers:

not used yet: 6/25
works: 3/25
great: 10/25
bad: 4/25
ugly: 2/25

Community GIT repository for eZ Publish

nospam@example.com (Thomas Koch) — Wed, 11 Mar 2009 19:51:42 +0100

The eZ Publish Community came together in january to the International Winter Conference in Barcelona. One issue which has been discussed often at this meeting was how the development of eZ Publish could become more open.
As part of a possible solution, I presented the distributed version control system GIT. A poll made in preparation of the conference, has pointed out that half of the respondents already uses a modified version and therefor a kind of fork of eZ Publish. This number has also been confirmed by the conference participants.
As a result Kristof Coomans, Kaarel Jõgi, and me think, that it is about time, to provide a community GIT repository of eZ Publish and thus allow everybody to work on things like:

make the default install of eZ Publish usable in more community related aspects: forums, blogs. This includes mostly templates and permission setup (no kernel modifications) and help people to get started with eZ Publish.
improve the update mechanism of eZ Publish
replace ezwebin templating setup with something more sensible and more easily themeable, maybe look into blueprint css or some similiar
AJAXify the admin area
include functionality from eZ Components:
- ezcTemplate engine in parallel to old eZ Publish templating
- ezcConfiguration for ini file handling
- ezcDatabase for query abstraction
- ezcAuthentication to allow more authentication mechanisms (OpenID!)

To make it possible, that feature branches can be included in an eZ Publish release, it would be suggested to respect the development guidelines of eZ Systems. These affect not only the implementation itself, but also the development process. The most important characteristic of this process is the split up into these steps:

Requirements Discussion
Design Discussion
Test Driven Implementation
Review

All of this steps need to be done openly on the development mailing list.
My employer, YMC was so kind to provide my working time and server ressources and let me setup a git-svn clone of eZ Publish and an issuetracker.

Why Zend Framework sucks. Part 1/infinite - Exceptions Missuse

nospam@example.com (Thomas Koch) — Thu, 26 Feb 2009 16:27:20 +0100

This is the first entry in a series of posts explaining, why you should never ever consider using Zend Framework. I write these entries, because I see many people considering Zend Framework, without knowing how bad it is and without knowing, that there are better alternatives. I'm very fluent in eZ Components so this is the alternative I propose and use as comparission. Please feel free to add your beloved PHP Framework or Library in the comments, but only when you point out, how it is better then ZF. Tobias Schlitt once found this (or similiar) code:

CODE:

try { $username = $this->_getAccountDn($username); } catch (Zend_Ldap_Exception $zle) { /** * @todo Temporary measure to deal with exception thrown for ldap extension not loaded */ if (strpos($zle->getMessage(), 'LDAP extension not loaded') !== false) { throw $zle; } // end temporary measure switch ($zle->getCode()) { case Zend_Ldap_Exception::LDAP_NO_SUCH_OBJECT: case Zend_Ldap_Exception::LDAP_X_DOMAIN_MISMATCH: throw $zle; } throw new Zend_Ldap_Exception(null, 'Failed to retrieve DN for account: ' . $zle->getMessage(), Zend_Ldap_Exception::LDAP_OPERATIONS_ERROR); }

From: trunk/library/Zend/Ldap.php, Line 689

Things to note:

The message of an exception is checked to control program flow. This breaks encapsulation, since the message belongs to the exception class. It would have been better to make a subclass of the exception class and then use different catch clauses for different exception classes.

The comment says, that this has been a temporary measure. But the programmer should have known, that there isn't something like a temporary measure in programming. Once you've introduced a bad thing, you have to live with it for a very long time. Subversion also states, that this temporary measure has been introduced before January 2008.

Update: Found the appropriate statement in the ZF Coding Standard forbidding the above code:

Don't put important diagnostic information only in the text of the exception method. Create methods and members in derived exception classes as needed, to provide information to the catch block. Create an exception constructor method that takes appropriate arguments, and populate the members of the class with those arguments.

Webtreff: Content Storage

nospam@example.com (Thomas Koch) — Wed, 05 Nov 2008 15:42:56 +0100

Monday we had the topic of content storage (pdf) technologies at our webtreff meeting. We talked about the content model of eZ Publish, about ideas how it could be improved together with some general database design topics.
We already looked at two other approaches: The Java Content Repository and CouchDB.

Especially CouchDB appealed much to the folks and Benjamin offered to prepare a talk about it for dec/01. The next meeting will be on nov/17, 18:30 about Typo3. For more informations you can join our mailing list.

YMC GIT Wiki

nospam@example.com (Thomas Koch) — Tue, 07 Oct 2008 09:23:59 +0200

At YMC we frequently use wikis for client projects. The wiki then serves as whiteboard for specification and documentation. Normally we use MediaWiki, the engine that powers Wikipedia, for this.
But I personally don't like MediaWiki that much, because it's too feature rich for our purpose, it needs a Database and I'd rather use another Markup Language then the default Wikipedia language. So I searched for an alternative and found WiGit, a very basic PHP wiki which uses my beloved GIT as storage backend. Unfortunately WiGit is still written in a rather ugly PHP4 style so I forked it and am now moving it step by step to use eZ Components.

So far it uses:

ezcTemplate
ezcConfiguration
ezcUrl

The next steps are to integrate ezcAuthenticate to use different means of login, most importantly OpenId, ezcDocument to provide different wiki markups or restructured text and the recently released ezcMvcTools to clean up the index.php.

I'm doing this project for fun together with my collegue Emanuel. If you like to join us, you can find the GIT repository at http://git.koch.ro/?p=ymcgitwiki;a=summary and clone via

CODE:

git clone git://git.koch.ro/git/ymcgitwiki

There is also an Ohloh page for our litte toy project.

Lightning Talk at Google about VIM

nospam@example.com (Thomas Koch) — Thu, 02 Oct 2008 20:32:48 +0200

I'm sitting at Google's office in Zürich, participating at their Open-Source-Jam. I had the pleasure to meet Bram Moolenaar, the creator of VIM and to give a short lightning talk about VIM as your (PHP) IDE. For those looking for the scripts, I can direct you for now to the GIT repository of the VIM-debugger (that works with xdebug) and the debian mentors site for the packaged script.

The rest of my scripts will be packaged soon. You can quicken my effort by writing a comment to this blogpost and saying that you're waiting.

Presentation Draft: VIM as your PHP IDE

nospam@example.com (Thomas Koch) — Tue, 02 Sep 2008 08:01:33 +0200

Hans Melis asks for a good PHP IDE and Tobias Schlitt will present PHP development with VIM at the Köln-Bonn PHP User-Group.

Since I also plan to make a presentation about VIM, I thought I could publish the draft of my presentation so far and kindly ask you to share your ideas about other cool things that VIM does for you.