GarfieldTech - Technical thoughts, tutorials, and musings View RSS

It's amazing what you can do when someone is willing to pay for the time!

There have been two new releases of Crell/Serde recently, leading to the latest, Serde 1.5. This is an important release, not because of how much is in it but what major things are in it.

That's right, Serde now has support for union, intersection, and compound types! And it includes "array serialized" objects, too.

mixed fields

A key design feature of Serde is that it is driven by the PHP type definitions of the class being serialized/deserialized. That works reasonably well most of the time, and is very efficient, but can be a problem when a type is mixed. When serializing, we can just ignore the type of the property and use the type of the value. Easy enough. When deserializing, though, what do you do? In order to support non-normalized formats, like streaming formats, the incoming data is opaque.

The solution is to allow Deformatters to declare, via an interface, that the can derive the type of the value for you. Not all Deformatters can do that, depending on the format, but all of the array-oriented Deformatters (json, yaml, toml, array) are able to, and that's the lion's share of format targets. Then when deserializing, if we hit a mixed field, Serde delegates to the Deformatter to tell it what the type is. Nice.

Sometimes that's not enough, though. Especially if you're trying to deserialize into a typed object, just knowing that the incoming data is array-ish doesn't help. Serde 1.4 therefore introduced a new type field for mixed values: #[MixedField]. MixedField takes one argument, $suggestedType, which is the object type that should be used for deserialization. If the Deserializer says the data is an array, then it will be upcast to the specified object type.

class Message
{
    public string $message;
    #[MixedField(Point::class)]
    public mixed $result;
}

When serializing, the $result field will serialize as whatever value it happens to be. When deserializing, scalars will be used as is while an array will get converted to a Point class.

Unions and compound types

PHP has supported union types since 8.0, and intersection types since 8.1, and mixing the two since 8.2. But they pose a similar challenge to serialization.

The way Serde 1.5 now handles that is to simply fold compound types down to mixed. As far as Serde is concerned, anything complex is just "mixed," and we just defined above how that should be handled. That's... remarkably easy. Neat.

If the type is a union, specifically, then there's a little more we can do.

First, if a union type doesn't specify a suggestedType but the value is array-ish, it will iterate through the listed types and pick the first class or interface listed. That won't always be correct, but since the most common union type will likely be something like string|array or string|SomeObject, it should be sufficient in most cases. If not, specifying the $suggestedType explicitly is recommended.

Second, a separate #[UnionField] attribute extends MixedField and adds the ability to specify a nested TypeField for each of the types in the list. The most common use for that would be for an array, like so:

class Record
{
    public function __construct(
        #[UnionField('array', [
            'array' => new DictionaryField(Point::class, KeyType::String)]
        )]
        public string|array $values,
    ) {}
}

In this case, if the deserialized value is a string, it gets read as a string. If it's an array, then it will be read as though it were an array field with the specified #[DictionaryField] on it instead. That allows upcasting the array to a list of Point objects (in this case), and validating that the keys are strings.

Improved flattening, now for the top level

Another unrelated but very cool fix is a long-standing bug when flattening array-of-object properties. Previously, their type was not respected. Now it is. What that means in practice is you can now do this:

[
	{x: 1, y: 2},
	{x: 3, y: 4},
]

class PointList
{
    public function __construct(
	    #[SequenceField(arrayType: Point::class)]
	    public array $points,
    ) {}
}

$json = $serde->serialize($pointList, format: 'json');
$serde->deserialize($json, from: 'json', to: PointList::class);

Boom. Instant top-level array. Previously, this behavior was only available when serializing to/from CSV, which had special handling for it. Now it's available to all formats.

New version requirements

Because compound types were only introduced in PHP 8.2, Serde 1.5 now requires PHP 8.2 to run. It will not run on 8.1 anymore. Technically it would have been possible to adjust it in a way that would still run on 8.1, but it was a hassle, and according to the Packagist stats for Crell/Serde the only PHP 8.1 user left is my own CI runner. So, yeah, this shouldn't hurt anyone. :-)

Special thanks

These improvements were sponsored by my employer, MakersHub. Quite simply, we needed them, so I added them. One of the advantages of eating your own dogfood: You have an incentive to make it better.

Is your company using an OSS library? Need improvements made? Sponsor them. Either submit a PR yourself or contract the maintainer to do so, or just hire the maintainer. All of this great free code costs time and money to make. Kudos to those companies that already do sponsor their Open Source tool chain.

I am pleased to report that my latest side project, MiDy, is now available for alpha testing!

MiDy is short for Mildly Dynamic. Inspired by this blog post, MiDy tries to sit "in between" static site generators and full on blogging systems. It is optimized for sites that are mostly static and only, well, "mildly dynamic." SMB websites, blogs, agency sites, and other use cases where frankly, 90% of what you need is markdown files and a template engine... but you still need that other 10% for dynamic listings, form submission, and so on.

MiDy offers four kinds of pages:

1. Markdown pages. These should be familiar to anyone that's worked with any "edit file on disk" publishing tool before.
2. Latte templates. Latte is used as the main template engine for MiDy, but you can also create arbitrary pages as Latte templates. Want to have a one-off page where you control the HTML and CSS, but still inherit the overall page layout and theme? Great! Put a Latte template file in your routes folder and you're done.
3. Static files. Self-explanatory.
4. PHP. For the few cases where you need custom logic, you have a PHP class that can do whatever you'd like. It's still pathed based on the file system, just like any other file, but you get separate handlers for each HTTP method; and you get full arbitrary DI support as well.

The README covers more details, though as it's still at version 0.2.0 the documentation is still a work in progress. And of course, it's built for PHP 8.4 and takes full advantage of many new features of the language, like property hooks and asymmetric visibility. Naturally.

I will be converting this site over to MiDy soon. Gotta dog-food my own site, of course. (And finally get rid of Drupal.)

While I wouldn't yet recommend it as production ready, it's definitely ready for folks to try out and give feedback on, and to run test sites or personal sites on. I don't expect any API changes that would impact content at this point, but like I said, it's still alpha so caveat developor.

If you have feedback, please either open an issue or reach out to me on the PHPC Discord server. If you want to send a PR of your own, please open an issue first to discuss it.

I'll be posting more blog posts on MiDy coming up. Whether before or after I move this site to it, we'll see. :-)

I've long kept my photo backups off of Google Cloud. I've never trusted them to keep them safe, and I've never trusted them to not do something with them I didn't want. Like, say, ingest them into AI training without telling me. (Which, now, everyone is doing.) Instead, I've backed up my photos to my own Nextcloud server, manually organized them, and let them get backed up from there.

More recently, I've decided I really need a proper photo album tool to carry around "wallet photos" of family and such to show people. A few years back I started building my own application for that in Symfony 4, but I ran into some walls and eventually abandoned the effort. This time, I figured I'd see what was available on the market for self-hosted photo albums for me and my family to use.

Strap yourself in, because this is a really depressing story (with a happy ending, at least).

I reviewed 7 self-hosted photo album tools, after checking various review sites for their top-ten lists. Of those 7:

• 3 were in PHP, 2 were in JavaScript or TypeScript, and 2 were in Go.
• 2 used the MIT license, 2 used the GPL, 1 used AGPL, and two had broken non-free licenses.
• I managed to get one working. 1. Uno.
• Most really pushed you to use their Docker Compose setup to install, none of which actually worked.

Let's have a look at the mess directly.

PiGallery 2 (https://bpatrik.github.io/pigallery2/)

Language: TypeScript License: MIT

PiGallery 2 is intended as a light-weight, directory-based photo album. The recommended way to install it is to use their Docker compose file and nginx conf file... which you have to just manually copy out of Git. (Seriously?) And when I tried to get that to run locally, I could never connect to it successfully. There was something weird with the port configuration, and I wasn't able to quickly figure it out. If I can't get the "easy" install to work, I'm not interested.

Piwigo (https://piwigo.org/)

Language: PHP/MySQL License: GPLv2

Unlike many on here it doesn't provide a Docker image, which is fine so I set one up using phpdocker.io. Unfortunately, it's net installer crashed when I tried to use it, without useful errors. Trying to install manually resulted in PHP null-value errors from the install script. When I looked at the install script, I found dozens upon dozens of file system operations with the @ operator on them to hide errors.

At that point I gave up on Piwigo.

Coppermine (https://coppermine-gallery.net/)

Language: PHP/MySQL License: GPL, version unspecified

When I first visted the Coppermine website, I got an error that their TLS certificate had expired a week and a half before. How reassuring.

Skipping past that, I was greeted with a website with minuscule text, with a design dating from the Clinton presidency. How reassuring.

Right on the home page, it says Coppermine is compatible all the way down to PHP 4.2, and supposedly up to 8.2. For those not familiar with PHP, 4.2 was released in 2002, only slightly after the Clinton presidency. PHP has evolved, um, a lot in 22 years, and most developers today view PHP 4 as an embarrassment to be forgotten. If their code is still designed to run on 4.2, it means they're ignoring literally 20 years of language improvements, including security improvements. How reassuring.

Oh, and the installation instructions, linked in the menu, are a direct link to some random forum post from 2017. How reassuring.

At this point I was so reassured that I Noped right out and didn't even bother trying to install it.

Lemorage (https://lomorage.com/)

Language: JavaScript. (Not TypeScript, raw JS as far as I can tell.) License: None specified.

Although this app showed up on a few top-ten lists, its license is not specified, and installation only offers Windows and Mac. (Really?) The "others" section eventually lets you get to an Ubuntu section, where their recommendation is to install it via... an Apt remote. Which is an interesting choice.

It has a GitHub repo, but that has no license listed at all. Which technically means it's not licensed at all, and so downloading it is a felony. (Yes, copyright law is like that.)

Being a good Netizen, I reached out to the company through their Contact form to ask them to clarify. They eventually responded that, despite some parts of the code being in public GitHub repos, none of it is Open Source.

Noping right out of that one.

PhotoPrism (https://www.photoprism.app/)

Language: Go License: It's complicated

I actually managed to get this one to run! This one also "installs" via Docker Compose, but it actually worked. This is the only one of the apps I reviewed that I could get to work. Mind you, as a Go app I cannot fathom why it needs a container to run, since Go compiles to a single binary.

Their system requirements are absurdly high. Quoting from their site, "you should host PhotoPrism on a server with at least 2 cores, 3 GB of physical memory,1 and a 64-bit operating system." What the heck are they doing? It's Go, not the JVM.

In quick experimentation, it seemed decent enough. The interface is snappy and supports uploading directly from the browser.

However, I then ran into a pickle. The GitHub repository says the license is AGPL, which I am fine with. However, in the app itself is a License page that is not even remotely close to Free Software anything, listing mainly all the ways you cannot modify or redistribute the code.

I filed an issue on their repository about it, and got back a rather blunt comment that only the "Community Edition" is AGPL, which is a different download. The supported version is not.

Noping right out of this one, too.

Photoview (https://photoview.github.io/)

Language: Go, with TypeScript for front-end License: AGPLv3

Another app wants you install via Docker Compose. And when I tried to do so, I got a bunch of errors about undefined environment variables. The install documentation says nothing about setting them, and it's not clear how to do so, so at this point I gave up.

Lychee (https://github.com/LycheeOrg/Lychee)

Language: PHP License: MIT

Lychee is built with Laravel, which I don't care for but I have used very good Laravel-based apps in the past so I had high hopes. It talks about using Docker, but unlike the others here doesn't provide a docker-compose file, just some very long Docker run commands.

Their primary instructions are to git-clone the project, then run composer install and npm. Unfortunately, phpdocker.io is still built using Ubuntu 22.04, which has an ancient version of npm in it, and I didn't want to bother trying to figure out how to upgrade it.

Lychee did offer a demo container, which uses SQLite. That I was able to get to run successfully. However, for unclear reasons it wouldn't actually show any images.

At this point, I gave up.

So what now?

Rather disappointed in the state of the art, I decided to take a different approach. As I mentioned, I use Nextcloud to store all my images. Nextcloud has a photo app, but the last time I used it, it was very basic, and pretty bad. That was a few years ago, though, so I went searching.

Turns out, not only has Nextcloud Photos improved considerably, there's also another extension app on it called Memories. On paper, it looks like it does everything I'm after. A timeline feed, custom albums that don't require duplicating files, you can edit the Exif data of the image to show a title and description, plus some fancy extras like mapping geo information to OpenStreetMap and AI-based tagging, if you have the right additional apps installed. So would it work?

Turns out... yes. The setup was slightly fiddly, but mostly because it took a while to download all the map data and index a half-million photos. Once it did that, though... it just worked. It does almost everything I was looking for. I haven't figured out how to reorder albums or pictures within an album, and it looks like it doesn't support sub-albums. But otherwise, it does what I need. It even has a mobile app (free) that let's me show off selected pictures on my phone, which is what I was ultimately after.

I have always had a love/hate relationship with Nextcloud. In concept, I love it. Self-hosted file server and application hub? Sign me up! Despite being a PHP dev of 25 years, I've never quite understood why PHP made sense for it, though. And upgrades have always been a pain, and frequently break. But its functionality is just so useful. Apps are hit or miss, ranging from first-rate (like Memories) to meh.

But in this case, it ended up being both the cleanest and most capable option, as well as the easiest to get going, provided I already had a Nextcloud server. So, solution found. I am now a Memories user, and will be setting up accounts for the rest of the family, too.

Two of the biggest features in the upcoming PHP 8.4 are property hooks and asymmetric visibility (or "aviz" for short). Ilija Tovilo and I worked on them over the course of two years, and they're finally almost here!

OK, so now what?

Rather than just reiterate what's in their respective RFCs (there are many blog posts that do that already), today I want to walk through a real-world application I'm working on as a side project, where I just converted a portion of it to use hooks and aviz. Hopefully that will give a better understanding of the practical benefits of these tools, and where there may be a rough edge or two still left.

One of the primary use cases for hooks is to not use them: They're there in case you need them, so you don't need to make boilerplate getter/setter methods "just in case." However, that's not their only use. They're also really nice when combined with interface properties, and delegation. Let's have a look.

Continue reading this post on PeakD

I've just released version 2.0 of Crell/Tukio! Available now from your favorite Packagist.org. Tukio is a feature-complete, easy to use, robust Event Dispatcher for PHP, following PSR-14. It began life as the PSR-14 reference implementation.

Tukio 2.0 is almost a rewrite, given the amount of cleanup that was done. But the final result is a library that is vastly more robust and vastly easier to use than version 1, while still producing near-instant listener lookups.

Some of the major improvements include:

• It now uses Topological sorting internally, rather than priority sorting. Both are still supported, but the internal representation has changed. The main benefits are cycle detection and support for multiple before/after rules per listener.
• The API has been greatly simplified, thanks to PHP 8 and named arguments. It's now down to essentially two methods -- listener() and listenerService(), both of which should be used with named arguments for maximum effect. The old API methods are still supported, but deprecated to allow users to migrate to the new API.
• Tukio can now auto-derive more information about your listeners, making registration even easier.
• It now uses the powerful Crell/AttributeUtils library for handling attribute-based registration. That greatly simplified a lot of code while making several new features easy.
• Attributes are now supported on the class level, not just method. That makes building single-method listener services trivially easy.

Continue reading this post on PeakD.

Static methods and properties have a storied and controversial history in PHP. Some love them, some hate them, some love having something to fight about (naturally).

In practice, I find them useful in very narrow situations. They're not common, but they do exist. Today, I want to go over some guidelines on when PHP developers should, and shouldn't, use statics.

In full transparency, I will say that the views expressed here are not universal within the PHP community. They do, however, represent what I believe to be the substantial majority opinion, especially among those who are well-versed in automated testing.

Continue reading this post on PeakD.

I am pleased to announce that the trio of libraries I built while at TYPO3 have now reached a fully stable release. In particular, Crell/Serde is now the most robust, powerful, and performant serialization library available for PHP today!

Serde is inspired by the Rust library of the same name, and driven almost entirely by PHP Attributes, with entirely pure-function object-oriented code. It's easy to configure, easy to use, and rock solid.

For a full overview, I gave a presentation at Longhorn PHP 2023 that went into its capabilities in detail. Even then, I didn't have time to cover everything! Have a look at the README for a complete list of all the options and features available.

Serde is backed by two other libraries:

• Crell/fp is a simple functional programming utility library, mainly aimed at enabling functional pipes.
• Crell/AttributeUtils is a fully-featured attribute management library that builds on PHP's native attributes and adds a metric ton of functionality. A lot of the functionality of Serde is driven directly by AttributeUtils.

Give all three a try, and see how powerful modern PHP has become!

The term "technical debt" gets thrown around a lot. Way too much, in fact. Part of that is because it has become a euphemism for "code I don't like" or "code that predates me." While there are reasons to dislike such code (both good and bad), that's not what the term "technical debt" was invented to refer to.

So what does it mean? There's several different kinds of "problematic code," all of which come from different places.

Continue reading this post on PeakD.

In the last 2 years or so, I've run into a number of projects that claim to use the PSR-3 logging standard as published by the PHP Framework Interoperability Group (PHP-FIG, or just FIG). Unfortunately, it's quite clear that those responsible for the project have not understood PSR-3 and how it is intended to work. This frustrates me greatly, as PSR-3's design addresses a number of issues that these projects are not benefiting from, and it reduces interoperability between projects (which was the whole point in the first place).

Rather than just rant angrily online (fun as it is, it doesn't actually accomplish anything), many of my PHP community colleagues encouraged me to blog about using PSR-3 properly. So, here we are.

The Short, Short version

If you just want the final point, here it is.

If you're writing this:

$logger->info("User $userId bought $productName");

Then you're doing it wrong, abusing PSR-3, and may have a security attack vector. You need to switch to doing this instead:

$logger->info("User {userId} bought {productName}", [
   'userId' => $userId,
   'productName' => $productName,
]);

And if your logger isn't handling that properly, it means you have it misconfigured and need to fix your configuration.

If your project's documentation is telling you to do the first one, then your project's documentation is wrong, and it should be fixed.

If you want to understand why, read on.

Continue reading this post on PeakD.

Like much of the world I've been working to migrate off of Twitter to Mastodon and the rest of the Fediverse. Along with a new network is the need for new automation tools, and I've taken this opportunity to scratch my own itch and finally build an auto-posting bot for my own needs. And it is, of course, available as Free Software.

Announcing Mastobot! Your PHP-based Mastodon auto-poster.

Continue reading this post on PeakD.