PHP-GTK Community : a community site for PHP-GTK

When I was in school, passing notes took some effort. First, you needed to find a piece of paper that was large enough for your message, but small enough that it could be folded into the requisite football shape. Next, you had to write something. Anything smaller than several sentences just wasn’t worth the overhead, so you had to write about half a page’s worth of stuff, or draw a picture large and detailed enough to make it worth it. After that, you set about the process of folding your note into the aforementioned form. Finally, you had to negotiate with your neighbor to get the note from your desk to its final destination. All that was just to send the message. On the receiving side, the note was unfolded and read. Then your counterpart would go about constructing a response, refolding the note, and negotiating the return trip.

http://www.flickr.com/photos/kmorely

Passing notes in class was a task that required effort, skill and time. You sent a message and you waited for a response. If you thought of something new that you really needed to say, you had to wait until the response came back. At that point, you could alter your original message or add new content. While your note was in transit or being read and replied to on the other end, you had no control. You were at the mercy of the medium over which you were forced to communicate. Note passing simply isn’t designed to allow for short, quick, asynchronous communication.

Nowadays, kids just text each other on their smartphones. They send messages quickly and easily without having to invest in all that overhead. After a bit of upfront work to get someone’s phone number, the back channel classroom chatter flows freely. That is, until someone forgets to silence their phone and the teacher confiscates everything with a battery.

Just as the methods of slacking off in school have evolved, so have methods of communicating over the Web. HTTP is the note passing of the Internet. It works well enough for most communications, and when the message is large enough, the overhead is minimal. However, it is less efficient for smaller messages. The headers included by browsers these days can easily outweigh the message body.

Also, just like note passing, HTTP is synchronous. The client sends a request and waits until the server responds. If there is something new to be said, a new request is initiated. If the server has something to add, it has to wait until it is asked. It can’t simply send a message when it is ready.

WebSockets are the smartphone to HTTP’s notes. They let you send information quickly and easily. Why go through all that folding when you can simply send a text to say “idk, wut do u think we should do?” Why use 1K of headers when all you want to know is, “Did someone else kill my rat?” Better yet, why ask at all? Why not have the server tell you when the rat has been killed by someone else?

WebSockets are made for small messages. They are made for asynchronous communications. They are made for the types of applications users expect these days. That’s why I like WebSockets so much. They let me communicate without overhead or rigorous process. I can write an application that is free from request/response pairs. I can write an application that responds as quickly as my users can act. I can write the applications that I like to write.

Code is the way in which humans tell computers what to do. Lots of effort has gone into making code easier for humans to read in the form of high level languages like Java or C++ and scripting languages like PHP, Ruby, and Python. Despite mankind’s best efforts, writing code is still clearly an exercise for talking to computers. It has not evolved to the point where talking to a computer is as easy and natural as talking to other people.

That’s why documentation is so important. Programming languages are just a translation of a developer’s intent into something a computer can execute. The code may show the computer what you intended for it to do, but the context is lost when another developer comes back to it later. Computers don’t know what to do with context. If they did, the days of Skynet would already be upon us. Humans can process context and it makes the process of dissecting and understanding a computer program much easier.

I find it both sad and hilarious when I see a speaker evangelizing code without comments. Invariably, the speaker shows a slide with five lines of code and spends ten minutes explaining its form and function. Even the simplest and most contrived examples from some of the foremost experts in the field require context and explanation.

When a bug decides to show itself at three in the morning, in code that someone else wrote, context and intent are two very powerful tools. When bugs are found the question, “What was this supposed to do?” is more common than “What is this thing doing?” Figuring out what it is doing is easier when you have good log data to go on. Knowing what it was supposed to do is something only the original developer can tell you.

If you aren’t aware of the concept of Test Driven Development, I strongly recommend you dig into it. In summary, tests are written before the code to ensure that they code matches the business requirements. I would like to propose a complimentary development driver: Documentation Driven Development. By writing out the code as comments first, you can ensure that the context of the development process will be captured. For example, I start writing code with a docblock like this:

/** * Returns the array of AMQP arguments for the given queue. * * Depending on the configuration available, we may have one or more arguments which * need to be sent to RabbitMQ when the queue is declared. These arguments could be * things like high availability configurations. * * If something in getInstance() is failing, check here first. Trying to declare a * queue with a set of arguments that does not match the arguments which were used * the first time the queue was declared most likely will not work. Check the config * for AMQP and make sure that the arguments have not been changed since the queue * was originally created. The easiest way to reset them is to kill off the queue * and try to recreate it based on the new config. * * @param string $name The name of the queue which will be used as a key in configs. * * @return array The array of arguments from the config. */

Next I dive into the method body itself:

private static function _getQueueArgs($name) { // Start with nothing. // We may need to set some configuration arguments. // Check for queue specific args first and then try defaults. We will log where we // found the data. // Return the args we found. }

After that, I layer in the actual code:

/** * Returns the array of AMQP arguments for the given queue. * * Depending on the configuration available, we may have one or more arguments which * need to be sent to RabbitMQ when the queue is declared. These arguments could be * things like high availability configurations. * * If something in getInstance() is failing, check here first. Trying to declare a * queue with a set of arguments that does not match the arguments which were used * the first time the queue was declared most likely will not work. Check the config * for AMQP and make sure that the arguments have not been changed since the queue * was originally created. The easiest way to reset them is to kill off the queue * and try to recreate it based on the new config. * * @param string $name The name of the queue which will be used as a key in configs. * * @return array The array of arguments from the config. */ private static function _getQueueArgs($name) { static::$logger->trace('Entering ' . __FUNCTION__); // Start with nothing. $args = array(); // We may need to set some configuration arguments. $cfg = Settings\AMQP::getInstance(); // Check for queue specific args first and then try defaults. We will log where we // found the data. if (array_key_exists($name, $cfg['queue_arguments'])) { $args = $cfg['queue_arguments'][$name]; static::$logger->info('Queue specific args found for ' . $name); } elseif (array_key_exists('default', $cfg['queue_arguments'])) { $args = $cfg['queue_arguments']['default']; static:$logger->info('Default args used for ' . $name); } // Return the args we found. static::$logger->trace('Exiting ' . __FUNCTION__ . ' on success.'); return $args; }

The final result is a small method which is well documented and little if any extra time to write.

Armed with data from logs, unit tests which ensure functionality, configurations to control execution, isolation switches to lock down features, and contextual information in the form inline documentation, the process of finding bugs becomes easier. LUCID code communicates as if it were a member of the development team. It does all the things you expect from a coworker. It talks, it makes commitments, it works around problems and keeps a record of both what it is doing and why it is doing it.

At some point in its life, every team will have a bad egg. If it isn’t a hiring mistake that brings someone in, it can be a business choice or financial change that sours (if even only temporarily) a previously great player. What separates poor and mediocre teams from the great ones is their ability to effectively quarantine the disgruntled teammate until such time as the problem can be corrected. If the group can’t isolate the individual and continue to function at a high level, they run the risk of completely imploding. An ineffective dynamic can spread and negatively impact the entire organization.

The same can be said of an application. Applications are a collection of individual components; they pull together the contributions of individuals to create something which is greater than the sum of its parts. If one piece breaks down, and the system is not prepared to put it in quarantine, the entire application can come crashing down. There are no effective means for isolating the application, if the application is the team member that turns sour. Once this happens, the disease spreads and the entire company is at risk.

The best way to stave off this infection is to prevent it at the source. Keep the application happy by making it more resistant to implosion. Take the team approach down one level, inside your code, and imagine features as team members. If one of piece of the application, such as user authentication, goes a little crazy, will that bring down your entire system? Hopefully, users that are already logged in can still make purchases or update their user information or share links with their friends. Don’t let users that are having trouble logging in ruin the experience for everyone else. That isn’t to say issues with authentication aren’t a major concern, but they should be isolated in such a way that people who are already in can still do what they came to do.

Code jerks (the features, classes and libraries that bring everything else down when they are upset) come in all sorts of shapes and sizes. There are code jerks that hold resources too long. These jerks normally don’t do much on their own, but when you have lots of users executing the same code at the same time, you essentially end up DoS’ing yourself. There are code jerks that assume they know how the rest of the system works. These jerks tend to halt execution and prevent the rest of the application from working around the error. There are code jerks that give up too easily. These jerks try something once and assume things can’t get any better so there is no point in retrying. The worst code jerks are a combination of the three.

Resource hogs are tough to find, because they typically need lots of friends before they are a noticeable problem. For example, setting a connection timeout to five seconds for a third-party service may seem reasonable. However, when the service is slow and it takes three seconds to connect, you could end up keeping your own connections open much longer than expected. Can your server hold each connection open for three seconds and still handle all of the incoming requests? Even if only some of your user requests need this service, you may run out of connections for the rest of your users. Being able to turn this particular feature off without impacting the rest of your site can help keep your application running and making money.

It isn’t always third-parties that cause trouble. Some times it could be your own database that slows down your application. Let’s say you forgot to add an index, or your data suddenly changed dramatically and your old indexes don’t work as well and your database has become unresponsive. Many applications (including most of the ones I have built in the past) assume that not being able to talk to a database is a fatal error. The database wrapper library may halt execution of the app or make it difficult for the application to properly handle the situation. I have since learned that there are lots of things you can do without database access. For instance, depending on your data and your business rules, you may be able to use slightly old cache data in place of fresh database data. If you isolate the your application from certain types of failures, you can let your code decide what is best. Your code should do everything it can to satisfy the user’s request to the best of your ability and be as resilient to failure as your business allows.

Sticking with our database example, let’s look at what you might do when the primary database isn’t available. Assuming you can’t use the cache as a back up database, what can you do? Can you read from a secondary? Can you log data to a temporary location to be written to the database when it comes back up? Is the primary database really 100% critical to satisfying the user’s request? With a bit of creativity you can probably find a solution that allows you to keep the site up while you work to fix the database problem. But even before you get to that point, how do you know you it is really necessary? This may come as a shock to some, but the Internet is unreliable. It tells you something is there one minute, and then that it is gone the next. The inverse is equally true. Just because you tried to connect to the database and couldn’t, doesn’t mean you need to switch to crisis mode. Maybe you were just a victim of the Internet being the Internet. Code that tries multiple (configurable) times to connect to a database isolates the rest of the application from the reliability issues inherent in communicating between systems.

Approaching software as a set of interconnected and unreliable services helps to create applications which stand up better in the face of less than ideal situations. Unfortunately, even when you have an application that communicates well with logs, has tests to verify functionality and uses configurations to help isolate features from one another, there will still be bugs. Armed with your logs and tests, you will still have to dive into code and make changes. The level of documentation in your code can either make this easier, or a nightmare. The next post will look at how documentation contributes to the maintenance of code.