The Terrible, Horrible, No-Good, Very-Bad Things I Do to PHP

Hi! I'm M.J., and I happen to be quite fond of Common Lisp. So, naturally, I'm here to talk about PHP and the, uh, "interesting" things I do to it.

We're going to cover a number of topics in this talk, broken into sections of about one per topic. You might actually prefer to think of it as a series of lightning talks all by the same guy.

It's only fair to warn you that some of this stuff has been in my head for a very long time, so if I'm not managing to get it out in a way you understand, feel free to ask questions as we go. I'll also stop for questions in between each major section of this talk, so you'll have a chance to ask about things before we jump to a completely different topic. Like I said, this is really a lot of little talks all mushed into one.

How many people here have used PHP before?

Some, not all.

Okay. I do assume a passing familiarity with some of the PHP things I talk about, so let's go ahead and cover some of the basics really briefly for those who aren't as familiar with PHP.

Really, every last one of you?

Great! Then the prereqs should go really quickly.

• Typehint specified before parameter name
• Names a class or instance
• $arg must be an instanceof that class, or will error

function f(TypeHint $arg) {
   // ...
}

roughly equivalent to…

function f($arg) {
   if (!($arg instanceof TypeHint)) {
      trigger_error(...);
   }
   // ...
}

• PHP has two different ways of dealing with problems
• Errors and Exceptions
• Throwing an exception unwinds the stack to an appropriate catch
  • Failure to catch terminates program
  • Exception handler runs at top level (after stack unwinds)
• Errors log message, terminate program
  • Error handler runs in place (leaves stack alone)
  • return true ignores error

• "Multiple inheritance is haaaaaaard"
• But everybody wants it, because it's useful
• So… automated copy-and-paste
• Share implementation across multiple classes

WebUI

POST …/controller/method arg1=val&arg2=val2

MobileAPI

POST/PUT /endpoint.php {"arg1":"val","arg2":"val"}

… let's get started!

If you think I'm crazy by the end of this, well, I can't entirely disagree. But everything I do is guided by a few fundamental thoughts.

• Correct and secure code should be easier to write than incorrect or insecure code.
• It is my responsibility as a library or language author to make this possible.
• It is more important for the interface to be simple than the implementation
• Corollary: It is better for my replacement to hate me than for everyone who merely uses my code to hate me.

If you're familiar with Richard Gabriel's "Worse-is-Better" essay, I tend to err more on the MIT Approach side. Basically, it's my job to make everybody else's job easier.

And because I take that mindset, I sometimes do things that other people might consider insane.

Take this, for example. It takes a specially-formatted string from a form POST or JavaScript and converts it into a DateTime object, or if the string is formatted incorrectly, it'll throw an exception.

Or this, which takes a value from a form POST, ensures the string represents an integer greater than zero, and if it does gives me an actual int inside the function. If it doesn't, again, it'll throw an exception.

But those are the end result. Getting to this point was something of an iterative process, and like most stories, it starts out innocently enough.

When I started at BettrLife, this is what a controller method looked like. Kinda ugly, error-prone, a pain to write, and a pain to use. I mean, To even know what arguments you can pass to this method, you have to read through the entire method. It doesn't look terribly painful here, but that ellipsis I'm using for brevity hides a potentially large amount of complexity you have to wade through just to find out if there's an optional argument, or what the types of 'b' and 'c' are.

That's terrible.

So I set out to make a better way. By making use of PHP's introspective capabilities, we can define controller methods that look like normal PHP methods. Even though this defines something callable from the browser, it's just as easy to call it from other PHP code, you can tell what arguments it takes without needing to read through the entire function, and it's just generally better.

This is not a new idea. Symfony does it in the PHP world, Hunchentoot does something similar in Lisp, and I'm sure there are many other languages with a library that supports something along the same lines.

But types are still an annoyance. Checking every argument is a pain, which means people are going to skip it. And it means callers still have to read through the method to know how to format the data they're passing in. Hopefully the argument name provides a pretty good contextual clue, but probably not always. Sometimes it might even suggest the wrong thing.

Wouldn't it be nice if we could specify the types of the values as typehints?

And using typehints to specify the types of arguments in our controller method might look something more like this. Now enforcing types is easy, so there's no excuse. You just type out the right word! Now knowing what each parameter expects is easy, because you don't have to read the code. And, unlike a doccomment, it can't possibly end up out of date, because it's actual code. It means something in your program.

The only problem is it doesn't work. PHP's typehints must be classes, they can't be arbitrary names. And some people would stop here.

But PHP happens to be conveniently flexible in this area. Failure to match a typehint in PHP triggers an E_RECOVERABLE_ERROR, which means I can write an error handler that grabs the typehint name, validates the argument using an arbitrary function defined for that typehint, and either continues on or throws an exception.

You can find some examples of this in the comments on the Type Hints page in the PHP documentation, so again, not a new idea.

As a quick example, this defines an int type which is either an actual int (important for those times you call a method from PHP), or something that looks like an int. Then it defines a positive_int type which inherits from the int type with the additional restriction that it is greater than zero. Then it defines an alias for positive_int called row_id, because it's nice to have very specific type names.

There's more code behind the scenes to tie in the error handlers and all that, of course, and we'll get to some of that shortly, but the type checking itself is fairly straightforward.

So now it's easy to define new types, it's easy to validate types, most of the periphery of defining a method is fairly straightforward. Now this code [flip back two slides quickly] can actually work.

But it's still not quite there. There are certain more structured types we'd like to be able to pass from a form POST or JavaScript into PHP. For example, it'd be really nice to be able to pass a date from the client and get a DateTime instance in the controller method. But of course, the only thing we can get from request parameters is a string.

And yet we'd like this to work. It's not officially documented as such, but PHP happens to provide references to function arguments when grabbing a backtrace, which means while within our error handler, we can modify the arguments passed to the function, allowing us to make this work.

So we get a specially-formatted string from the client, parse it out into a date, and return a new instance of DateTime. The machinery does its thing, and our function gets a shiny new object instead of a string. Sweet!

Unfortunately, due to "quirks" in PHP's implementation, it only works for required arguments. I'd call them bugs, but my use-case is not supported. What's the point of even triggering a recoverable error if recovering from that error is not supported?—but whatever. And here, we finally hit the boundary of what is possible under this particular implementation strategy.

So that's how we ended up with controller methods performing type conversions based on typehints. It's a somewhat perilous situation—it may stop working in PHP7, probably doesn't work in HHVM at all, and is nonetheless incredibly useful. Barring the optional argument caveat, it makes correct code both easy to write, and very succinct.

Are there any questions on the typehint shenanigans?

As I mentioned earlier, all those typehint shenanigans are implemented through an error handler. The trouble with implementing typehint shenanigans via the error handler mechanism in PHP is there can only be one error handler. This makes it hard to compose assorted bits of code that want to handle different error conditions. This bit of code wants to log something, that bit over there wants to convert errors into exceptions, some other bit wants the typehint shenanigans, and so on.

I like things to be nice and composable—a large monolithic function that handles everything isn't really my thing—so I built a way to dispatch to different error handlers based on error codes, and error strings, and handler priorities. It's not a lot of code, and it sure beats maintaining a handler stack yourself.

That dispatcher enables us to do things like this, where we add two different error handlers—one for the typehint shenanigans, and another which converts errors to exceptions. The exception converter is set to run after any other error handlers, so if those error handlers handle the condition, it won't run, and so only unhandled errors get converted.

This is handy. Now we can split things up into small pieces, and enable each piece as necessary.

Now, some of you might be wondering why bother? There seems to be a contingent of people who think everything should be an exception, that errors should be abolished entirely from PHP. Those people are wrong.

Exceptions are great for what they do, but they're very limited. In Common Lisp, exceptions are generalized into conditions, and throwing to signaling. But more important than the name change is the behavior difference. Signaling a condition (throwing an exception), handling that condition (try-catch), and unwinding the stack are separated out. We can do some of that without having to do all it. In other words, we can handle a condition without unwinding the stack all the way up to the handler code.

To make that clear by way of analogy, let's say we're at a checkout line at the grocery store. So we've got ourself as the customer, the clerk, and the bagger. We give our items to the clerk, the clerk scans them, and gives them to the bagger who bags them.

But I've digressed somewhat. If you're interested in the typehint or error handling shenanigans, those are available via the Elephander library, which you can find on GitHub or composer install. And if you're interested in learning more about Common Lisp conditions, I recommend Peter Seibel's book Practical Common Lisp.

So are there any questions on error handler dispatch or conditions or what-have-you?

When I got hired at BettrLife, I was the odd man out. All the devs had worked together at a previous company, so they knew each other really well, their strengths and weaknesses, what each person likes to do and not do. So I got to come into a team with a really well-established dynamic, and that dynamic is pretty great.

But that previous company built a monolithic C++ app. I was really the first guy on the team with any significant web experience. (I probably could have given an entire talk just on the security holes we've closed since I started because the web is such a labyrinth of security pitfalls.) So, to avoid beating my head against the wall unnecessarily, whenever I'm evaluating an old architectural decision, I tend to have to frame it in the mindset of "Would this have made sense if this were a monolithic C++ app?". And usually the answer is yes.

So there's no ORM. Makes some development harder, but okay. They're not a panacea, I can kind of see the logic there. But in the end, you have to return some kind of structure from the models, so they return arrays. They originally returned resultsets, but let's not get in to that.

Arrays are great. They're very flexible. You want some extra stuff attached to a row? Just slap it in there!

But arrays come with problems. The big one being there's nothing enforcing any kind of structure. So two different people writing two different model functions pulling data from the same tables can and do end up returning arrays with different key names for the same values. Heck, even the same programmer on different days or in different moods might do that.

This is a pain. Not just because it means lots of refactoring to make it all align, but because that refactoring won't entirely help. If you recall the slide about our basic architecture, all that data syncs down to the mobiles. Since we can't control if or when a user updates their app, all those crazy key names need to keep working even after you've excised them from the rest of the codebase.

To solve this problem, we came up with what we call a DataObject, which when you get down to it is basically just an array with a well-defined structure.

So here, for instance, we're defining a Nutrition type, which has an id, some nutrients which we pull in from a trait shared by numerous other dataobjects, and this fancy bit here which is used to exclude the 'id' during a foreach loop. So it'll iterate over the nutrients—calories, sodium, vitamin A, etc.—but not the id.

Or in this example, we've got a recipe ingredient, and it specifies that the key ingredient_id is equivalent to the key id, so setting or fetching the ingredient_id key will set or fetch the underlying id property.

DataObjects gives us quite a lot of control. We can separately control which properties are json-encoded, which are iterable, what keys are aliases for other properties, and maybe one or two other things I'm forgetting. Doccomment annotations might have worked here too, now that I think about it, but nobody thought of them at the time.

But all that flexibility can be hard to manage.

As soon as inheritance and traits get thrown into the mix, things go to hell. Because now to use a trait, you have to know whether it defines any aliases, or excludes any of its keys. Same for your parent. And you have to know how to combine all of that stuff.

This is tedious and error-prone. People are going to screw it up. Fortunately, Common Lisp provides us with an answer to this problem: method combinations!

When you call a method on a class, you call that class's method, which potentially calls the parent class's method, and so on up the inheritance chain. But whether it does, and when it does, is programmer driven. Method combination generalizes and automates that concept.

To steal Peter Seibel's explanation from his most excellent book "Practical Common Lisp":

"method combinations produce an effective method that contains the code of all the primary methods, one after another, all wrapped in a call to the function … that gives the method combination its name. … For example, a generic function that uses the + method combination will return the sum of all the results returned by its primary methods."

For example, here each class defines a function f() which returns a value. We've also got a function m() which acts as the public interface, and creates a method combinator. That combinator walks the class hierarchy calling each function f(), and then sums the values returned.

So we create an A instance, call m, and it calls f() in A and returns 1. We create an instance of B, call m, and looks—oh, no f in B, so nothing to do there—but there's an f in A, so call that, and return 1. Then we create an instance of C, call f in C get three. C uses T, T has an f, call f in T, get 2. So now I've got three and I've got two. C's parent B doesn't have an f, nothing to do. B's parent A has an f, call f in A, get 1. Sum them all together and we get 6.

If you're wondering what's up with the T_f(), that's the abstraction leaking. We namespace the trait functions so trait users don't have to resolve name conflicts. If that was just function f() in the trait, then PHP would get bitchy when using it in class C.

In pictorial form, that might look more like this.

So, like in the code from the previous slide or our diagram here, each class function returns a value, and our method combinator runs the method in each class and trait, and sums the results together.

Now, obviously method combinations aren't applicable in every scenario, but when they are they're a really handy tool to have available. It's really a little sad they don't exist in more languages, because being able to abstract away the cruft of "How do I combine this with all the other pieces of this same sort of information?" from "What part do I care about right here?" is just fantastic.

If you recall our DataObject example from earlier [flip back to Scary DataObject Example], is really ugly and painful. By using method combinations [next slide] it becomes much more palatable.

Each function is responsible for just the part applicable to the class or trait it's in, and no more. Combining the results from all those functions is somebody else's problem, the author of a particular class or trait doesn't have to worry about it.

This is much more manageable! Easier to read, easier to write, harder to screw up. Parts are vastly more composable, I'm decoupled from the implementation of any given trait I might want to use, and maintenance is easier because I don't have to refactor any class which uses my trait, or any subclasses, just to add in a method.

If I've convinced you method combinations are pretty great, you can use them in your own projects by making use of the SchmancyOO library, available on GitHub and composer installable. If you'd like to learn more about method combinations in Common Lisp, check out the book Practical Common Lisp.

Any questions on method combination?

But that's not the end of our DataObjects, oh no. Because what we found was that fairly often we want to pass different subsets of data to the mobile phones and the browser.

While we code do that via the mobile-API-specific code, or the web controllers, that ends up sucking, because you end up having to do it in multiple places, and now you no longer have a single place you can look to see what format a given object will be in. We'd quickly end up in the same situation we were in before, which this API gives you one thing, and that API gives you another.

We certainly don't want that.

So again I turned to Common Lisp for inspiration. There's a library ContextL which provides something called Context-Oriented Programming. You can kind of think of it as Aspect-Oriented Programming done better. The author Pascal Costanza actually wrote an aspect-oriented programming library for Common Lisp, and when he was done looked at it and realized it wasn't really necessary in Lisp. And that inspired him to invent Context-Oriented Programming.

Which, as it's name kind of implies, is a way of saying "within this context, apply this specific behavior". Add this piece, take away that, whatever.

For instance, here we define the structure for a message in a conversation thread. That user property is an automatically-populated nested object. If we send six messages from the same user, we'd send that user down the pipe six times. Redundant data might compress well, but it's still a waste of memory and bandwidth and battery life. That's okay for the web UI since it simplifies populating a template, but we don't really want to do that via the mobile API where data is precious. We don't want to be the app that drains the battery.

So we created what we call a Component, which is a little adjunct class that gets attached to our DataObject, and defines structural changes to that DataObject that apply only within a particular context.

This component just says that within the context of the mobile API, we exclude the 'user' key from JSON serialization. But components can do pretty much anything our DataObjects support: we could have just as easily added an alias that only exists within the context of the mobile API, or added a property that only exists within the context of the web, or what-have-you. There are lots of possibilities.

Or, if we put that in a diagram rather than code, we can specify that our object has certain base properties, and add or remove those properties depending on context. So we have our base ThreadMessage object, and inside the WebUI no changes are made so it comes out the same, but when we filter it through the mobile API that user key gets removed and it ends up looking a little different.

Activating a layer isn't particularly hard. Here I'm using a pattern from Lisp, with-foo, which takes a thunk and sets the context prior to calling the thunk and resets it back to whatever it was before returning.

Obviously this particular example is not realistic, but it demonstrates things in action. We start in the WebUI, json_encode our object, and get one thing, enter the MobileAPI, json_encode that same object, get something else, then back out into the WebUI and get what we got before. Our object itself never really changes, but the way it serializes does.

So far in our codebase, nobody really has to deal with this. It's already taken care of prior to entering the Mobile API or Web Controllers. But if we wanted to have more specific layers, say a layer for each specific version of the mobile API, or a layer for some narrow section of code, we could totally do that, it's not super hard.

Defining a layer can be a little trickier. Here, for example, we define the WebUI layer. It's disjoint with some of our other layers, so whenever it's activated it disables those layers. When we leave the scope that enabled the WebUI layer, anything it deactivated will be automagically restored.

Again, not something that tends to happen much in our codebase—the basic layers of MobileAPI and WebUI are already defined and we haven't had much call for additional layers—but still something we might want to do at some point.

There's probably some room to make defining a new layer easier, but for now this works well enough.

Our Context-Oriented stuff is not available as open source, but if you're interested in the general idea, check out ContextL. There's not a lot of documentation, but there is an overview paper which should give you a decent idea of what Context-Oriented Programming is all about.

Any questions on context-oriented programming?

NOTE: Check time! Skip if near end of time.

BettrLife is a "Health and Wellness" app. So we do a lot of things with nutrition. Which means we have a lot of templates that involve nutrients. They're everywhere!

Our basic nutrition structure is dead simple. This makes lots of things, like math operations, really easy. But the templates are incredibly repetitive.

And that just won't do. Any time you have to repeat yourself is a recipe for bugs and inconsistencies. Nevermind all the changes you'd have to make to add or remove a nutrient. It'd be crazy for every template involving nutrients to repeat all this data.

So maybe you reach for a ViewModel. You trade a much more complicated data structure for a much less repetitive template. And that certainly works.

But it sucks. Now every user of the template has to know to transform the data, even if they're merely incidental users. Passed data to a template that happens to import this template? You'd better have transformed the data! The necessity of the data transformation leaks out. The fact I have to remember to do a transformation at all makes it error prone. I'm a simple guy, I want to take data out of the model, shove it at the template, and call it a day.

It also complicates client-side templates. Your JavaScript already has that nice simple structure where each nutrient has a key, and it does something with that, and now you have to complicate it by making it transform that data before it can be put into a template.

So now my transformation's duplicated, my structure's complicated, template use got harder. This solution has some benefits I desperately want, but overall, it blows.

So instead, we go meta. We take the definitions, which are basically template data, and use that to populate a template. That template outputs a template which takes a nice simple data structure.

The term meta-template might be a little scary, but it's not really all that different from our ViewModel template. [flip back] They're very similar in appearance. So for this teensy bit of extra mental effort to go meta, we get the benefit of the brevity and ease of maintenance of our ViewModel template, with the simplicity of use of our original repetitive template.

And that, to me, is a pretty clear win.

Any questions on meta-templates?

Awesome. Well, thanks for listening!

This talk is available online, so if you'd like to go over it again or want links to more details, there's the URL. That's also got my e-mail address, so if you want to pester me about something, you can totally do that.

If I've managed to pique your interest in Common Lisp, check out the book "Practical Common Lisp" by Peter Seibel. It's available for free online, and you can also buy a dead tree version. Peter Seibel is a very lucid author and does a pretty good job relating the strange and wonderful things in Common Lisp to programming idioms with which you're probably already familiar, so it's totally worth your money.

Thanks for having me!