#composerphp

[doc-core-insight]

TYPO3 Extension dependencies revisited

tl;dr: Don't try to forsee the future, but add TYPO3 composer dependencies wisely

In my last article "composer.json specification for TYPO3 Extensions" I already specified in a few words, how the require section for extensions should look like.

Let's dissect some of the statements a bit further and add some more explanation and context to it.

The require property MUST be provided and it MUST contain the requirement to TYPO3 with a version constraint.

It should be very easy to understand, that TYPO3 extensions only work with TYPO3 and thus TYPO3 needs to be added as dependency to the composer.json of extensions. But let me elaborate what I recommend as concrete version constraint.

If your extension is compatible with the two latest LTS releases, the version constraint is pretty straightforward: ^6.2 || ^7.6

If your extension relies on a certain bugfix of a certain bugfix version, you can specify that minor release easily: ^6.2.13 || ^7.6.11

Things get a little bit more tricky, if your extension supports current TYPO3 sprint releases. It is important to know that minor versions bumps of TYPO3 sprint releases may contain breaking changes. If your extension is compatible with version 8.0.0, it does not necessarily mean it is compatible with 8.1.0 or 8.2.0.

If you now specify a version constraint of ^8.0.0 or < 8.99.99 at the time when version 8.0.0 is released, then you declare that this exact same version, this exact same code base, will be compatible with all future sprint releases, which in fact you can not, unless you can forsee the future.

If you nevertheless use such a loose version constraint, you are making a bet. If you are lucky, nothing bad happens and your code magically works with a future sprint release. If you are not so lucky, than your incompatible extension version might be installed with the latest TYPO3 sprint release and might be the reason, why a complete TYPO3 installation is broken.

I suggest not to bet, but specify exactly with which sprint release version your extension is compatible: ^7.6 || >=8.3.0 <8.6 would be a perfect version constraint, when 8.5.0 has been released and your extension version is compatible with that version, but needs at least 8.3.0. If 8.6.0 is released and your code still works, just bump the version constraint with one commit and release a new version that is compatible with the latest TYPO3 sprint release. If your code does not work, you made sure, broken code is not installed.

It SHOULD contain at least a requirement to typo3/cms-core. It SHOULD NOT contain a requirement to typo3/cms directly to avoid confusions as extensions cannot be used as root package with TYPO3.

Now what does this mean? The package typo3/cms-core isn't available on Packagist, so why would I want to add a dependency to it? Well, here is the deal: typo3/cms is available on Packagist and declares to be a replacement for typo3/cms-core and all other system extensions of the current TYPO3 core (typo3/cms-backend, typo3/cms-info, …). Currently if you specify typo3/cms-core, your extension composer.json file cannot act as a root package in the sense that you cannot go to the extension directory and simply type composer install. If you do so, you'll get an error message. This is important, because TYPO3 cannot deal with extensions that are not in the typo3conf/ext directory anway.

You may want to install and test multiple TYPO3 version in your CI setup. But even then I find it useful to not specify typo3/cms directly in the extension composer.json and do composer install during CI, because then composer will always only install the latest TYPO3 version your extension is compatible with.

So you would end up doing composer require typo3/cms $TYPO3_VERSION anyway, which would overwrite any previously specified dependency to typo3/cms, but will add a depedency to it alongside letting composer check whether the newly required TYPO3 version is actually compatible with what the extension specifies. Believe me or, this setup caught quite some errors for me and some other people I know. It is not a strict requirement to use typo3/cms-core instead of typo3/cms, but it gives enough benefits to recommend it.

It COULD contain requirements to other individual TYPO3 system extensions, to properly specify which TYPO3 extensions this extension depends on. This is useful especially in combination with TYPO3 Console, but also provides information, which will be useful in the future.

What I mean with that is, that individual core extensions might be split out of the big TYPO3 repository, making it possible to install only these TYPO3 system extensions, that are strictly required for TYPO3 or your project. For this to work properly, extension (or composer package) authors would need to start specifying which exact TYPO3 system extensions their code depends on. I did this for TYPO3 console for example. Doing so now comes with no downside and no side effect. It only enables better compatiblity for the future and allows TYPO3 Console to automatically detect which TYPO3 system extensions need to be active in a project.

Hope this clears some things up. Happy composer.json writing!

[phptutorialforyou]

@phpc : RT @el_stoffel: My favorite page of the #composerphp book is page 599 which lists all contributors. Do you spot yourself? https://t.co/dJG594ZdL7

[e-commerce-magento]

RT @seldaek: Composer v1.9 is out now! With https://t.co/6Bd0qIK6xJ and a few other minor bugfixes #composerphp

Composer v1.9 is out now! With https://t.co/6Bd0qIK6xJ and a few other minor bugfixes #composerphp

— Jordi Boggiano (@seldaek) August 2, 2019

from Twitter https://twitter.com/fbeardev

[e-commerce-magento]

RT @packagist: Check out this free extension our customers @Wizkunde made to integrate Magento with Private Packagist for Vendors! It lets their customers install purchased packages with #composerphp instead of having to download and manage zip files https://t.co/LyBfjzAWbg

Check out this free extension our customers @Wizkunde made to integrate Magento with Private Packagist for Vendors! It lets their customers install purchased packages with #composerphp instead of having to download and manage zip files https://t.co/LyBfjzAWbg

— Packagist (@packagist) April 1, 2019

from Twitter https://twitter.com/fbeardev

[e-commerce-magento]

RT @packagist: Announcing Private Packagist for Vendors: The most customer friendly way to distribute paid #PHP packages with #composerphp - Blog post https://t.co/evGk7yyIhJ & Feature page https://t.co/PAF6FqAzen

Announcing Private Packagist for Vendors: The most customer friendly way to distribute paid #PHP packages with #composerphp - Blog post https://t.co/evGk7yyIhJ & Feature page https://t.co/PAF6FqAzen

— Packagist (@packagist) May 20, 2019

from Twitter https://twitter.com/fbeardev

[e-commerce-magento]

RT @flamingek: Ok, let’s simplify my last question with a pool. You're working for a web agency which manages multiple PHP based customer projects and want to reuse private #ComposerPHP packages among those projects. Which package hosting for PHP are you using?

Ok, let’s simplify my last question with a pool. You're working for a web agency which manages multiple PHP based customer projects and want to reuse private #ComposerPHP packages among those projects. Which package hosting for PHP are you using?

— Antonio Mansilla (@flamingek) April 25, 2019

from Twitter https://twitter.com/fbeardev

[e-commerce-magento]

RT @packagist: Check out this free extension our customers @Wizkunde made to integrate Magento with Private Packagist for Vendors! It lets their customers install purchased packages with #composerphp instead of having to download and manage zip files https://t.co/LyBfjzAWbg

Check out this free extension our customers @Wizkunde made to integrate Magento with Private Packagist for Vendors! It lets their customers install purchased packages with #composerphp instead of having to download and manage zip files https://t.co/LyBfjzAWbg

— Packagist (@packagist) April 1, 2019

from Twitter https://twitter.com/fbeardev

[phptutorialforyou]

@phpc : RT @naderman: Going to be at #Symfony_Live Berlin 2018 this Friday! There are still a couple day tickets available if you're interested :-) Looking forward to chatting #composerphp and Private Packagist with you! https://t.co/tncGHtRUPP

[phptutorialforyou]

@phpc : RT @seldaek: https://t.co/5DAoYX0x91 just crossed 10 billion installs counted 🎉 https://t.co/62e7ZxRLBa #composerphp

[phptutorialforyou]

@phpc : RT @naderman: At the airport in Austin with @seldaek waiting for our flight to Las Vegas for #MagentoImagine following an amazing #longhornphp - looking forward to answering all the #composerphp questions and introducing more people to Private Packagist! #RoadToImagine

[phptutorialforyou]

@phpc : RT @seldaek: After triaging/merging/fixing almost 200 issues in the last couple days, Composer v1.6.4 is out! 😅 It contains a security fix and is therefore a much recommended update for all. 📝 https://t.co/SjAfaXzZRW #composerphp

[e-commerce-magento]

RT @seldaek: Composer 1.6 is on its way! Please test the release candidate with "composer self-update --preview" and report any regressions! https://t.co/BSy9XhymeE #composerphp

Composer 1.6 is on its way! Please test the release candidate with "composer self-update --preview" and report any regressions! https://t.co/BSy9XhymeE #composerphp

— Jordi Boggiano (@seldaek) December 19, 2017

from Twitter https://twitter.com/fbeardev

[e-commerce-magento]

RT @seldaek: Composer turns 6 today! Not such a special number, but it's still quite a long time to keep working on a single project :) #composerphp

Composer turns 6 today! Not such a special number, but it's still quite a long time to keep working on a single project :) #composerphp

— Jordi Boggiano (@seldaek) April 5, 2017

from Twitter https://twitter.com/fbeardev

[doc-core-insight]

composer.json specification for TYPO3 extensions

EDIT (13.08.2016, 22:30h): Added hints about "version" property and amended the "require" property description.

tl;dr: On top of general requirements for composer packages, TYPO3 extensions need some additional information in a composer.json file to function properly.

How to read this specification

I have created an example with a minimal setup for a TYPO3 extension. In the following I will describe each property in the composer.json using the modifiers MUST, SHOULD, COULD. MUST means that it will break hard without specifying that configuration property or if specifying a wrong value. SHOULD means that not providing that configuration property might lead to faulty or inconsistent and confusing behaviour. COULD means that this is completely optional, but a nice addition and therefore recommended.

The following rules apply in case the extensions are released publicly. For private extensions the requirements are less strict, but they SHOULD be followed nevertheless.

Configuration properties

"name"

The name property MUST be provided and it MUST have a vendor name and a package name (separated by a slash /). Vendor name and package name MUST only contain lowercase alphanumerical characters, dashes or underscores (regex of the complete name: [a-z0-9]([_.-]?[a-z0-9]+)*/[a-z0-9]([_.-]?[a-z0-9]+)*).

The vendor name MUST not already be registered on Packagist. If it is registered, you MUST be owner of this vendor name on Packagist. If it is not registered you MUST register at least one package with this vendor name on Packagist.

"description"

The description property is one line describing the extrension and MUST be provided.

"keywords"

The keywords property COULD be added to tag your extension with some keywords.

"homepage"

The homepage property COULD be added to provide an URL to the homepage of the extension.

"authors"

The authors property SHOULD be added to provide information who created the extension.

"license"

The license property SHOULD be added to specify the license under which the extension is published

"type"

The type property MUST be provided and it MUST exactly have the value typo3-cms-extension

"version"

The version property SHOULD NOT be provided, if you are using TYPO3 version 7.6 and above. However for TYPO3 version 6.2 it SHOULD be provided to avoid issues during extension activation in TYPO3 extension manager.

"require"

The require property MUST be provided and it MUST contain the requirement to TYPO3 with a version constraint. It SHOULD contain at least a requirement to typo3/cms-core. It SHOULD NOT contain a requirement to typo3/cms directly to avoid confusions as extensions cannot be used as root package with TYPO3. It COULD contain requirements to other individual TYPO3 system extensions, to properly specify which TYPO3 extensions this extension depends on. This is useful especially in combination with TYPO3 Console, but also provides information, which will be useful in the future.

For TYPO3 version 6.2, the require section MUST NOT contain any other dependencies than the TYPO3 version.

For TYPO3 version 7.6, the require section SHOULD NOT contain any dependencies than the TYPO3 version. If composer packages are added, then non composer usage of the extension SHOULD be considered. If TYPO3 extensions are added, then the composer name from Packagist SHOULD be used. If the dependent extension is only available in TER, then the TER composer name SHOULD be used.

"replace"

The replace property MUST be provided if the composer package name part does not match the extension name and is highly recommended thus SHOULD be provided in any case. If it is provided, the first replace value MUST match exactly the extension key and the second replace value SHOULD match the TER package key (extension key prefixed with "typo3-ter/" and with underscores (_) replaced with dashes (-). The version constraint MUST exactly be self.version

"autoload"

The autoload property MUST be provided if the extension provides PHP classes. The autoload definition MUST be psr-4 or classmap and MUST NOT contain some other autoload type. The folder name for psr-4 is recommended to be Classes/, but can be any other folder as well. The PHP namespace prefix for psr-4 MUST only have two parts, where the first MUST be your vendor name in upper camel case and the second MUST be your extension name in upper camel case. While the latter sounds very restrictive, it is essential especially when using Extbase, as the framework very often makes assumptions on the extension key derived from the namespace prefix.

Tools support

Currently there is no tool support, so you have to check the above specification manually. I would find it useful to have two things. One tool, that supports you with creating a composer.json for your TYPO3 extension. A second tool could validate existing composer.json files against the specification above. If you are willing to contribute such functionality, get in touch with me and I will mentor this effort. Thanks in advance.

Spread the word!

Please check your own extensions and fix the composer.json files accordingly and help others to fix their composer.json files. It'll help to improve the TYPO3 extension ecosystem.

Happy composering

[doc-core-insight]

How to use PHP libraries in legacy extensions

tl;dr: We would be better off fully embracing composer in the TYPO3 ecosystem. You should do so for your own projects.

If you are not interested in a little bit of history, you can safely skip the following chapter.

History

This is actually a pretty good question, but also a pretty old one. Interestingly this is nowadays asked much more frequently than a few years ago. And it is always asked by mentioning composer in some way.

Like: "I know how to include this PHP library if I would use composer, but how do get this to work for my TER extension?"

Or: "I would like to use this PHP library in my extension, but it must also work for people not using composer?"

I can only speculate, but I assume the reason that this question pops up so often now is, that composer has already transformed the way people in the PHP world think about code.

The years B.C.

In the years B.C. (before composer), PHP code was handled differently. It was randomly copied from forums, publicly available code repositories or at best taken from PEAR and then burried into an extension and most of the time kept there untouched for ages. Updates or bugfixes in libraries weren't handled most of the time and sometimes the library code was changed arbitrarily for the needs of the extension, making updates of the library code close to impossible. But it worked well (enough). It worked well enough because nobody knew a better way to handle it.

The age of composer

Then some crazy people decided to change the PHP world and created a set of tools and kickstarted an ecosystem which since then thrives. You need a PHP library in your composer managed project? Go find it on Packagist, type composer require name-of-vendor/name-of-package in your project root and be done. Using the library code just works. Updating it works simply by typing composer update name-of-vendor/name-of-package. If the library needs another library, or anything needs a special PHP version, or another library needs another version of a library you intend to use, composer will resolve everything for you just nicely (most of the times).

The TYPO3 Extension Repository (TER)

The TER exists for years and it an integral part of the ecosystem that made TYPO3 thrive. However technically it is a bit aged. The TER and the TYPO3 Extension manager can handle updates of TYPO3 versions and even extensions. It can also handle dependencies to other extensions. However resolving these dependencies is less stable and capable than what composer does. Most importantly it can only handle dependencies from the TYPO3 ecosystem, not from the composer ecosystem. This means, if you want to publish an extension to TER, that needs code from a composer library, you need to take care and include it into your TYPO3 extension, like in the old B.C. days.

Bundling libraries in phar files

There is a project, which you can use to bundle any composer based library, or even a set of libraries as single phar file. I find it much nicer to ship one file with dependent third party code, instead of hundreds of files.

Benefits

More secure: The complete phar might be exposed but not single files.

Cleaner: It is much more clear which is a library and which is your code.

More reliable: Code scans or code formatters or linters in your CI environment will ignore phar files by default

Smaller: The phar files are smaller in size than the sum of all files

And probably more, but these are enough to convince me.

Howto

Install the tool globally and make sure $HOME/.composer/vendor/bin is set in your path:

composer global require clue/phar-composer

Create a directory in your extension and create the phar archive (symfony/process for example):

mkdir Libraries phar-composer build symfony/process Libraries/symfony-process.phar

In your extension code, before you want to use the libraries in question, include the autoloader file of your phar:

use TYPO3\CMS\Core\Utility\ExtensionManagementUtility @include 'phar://' . ExtensionManagementUtility::extPath('ext-key') . 'Libraries/symfony-process.phar/vendor/autoload.php';

Optimization

There are several things I have done to optimize this in my TYPO3 Console package:

I excluded the phar from the packages delivered by composer. In that case the phar is not needed.

I recreate the lib with the current version on every travis build, which means the TER upload will always include the current version

I set classmap-authoritative to true and prepend-autoloader to false for the phar class loader. That way I can just include the phar if present and the autoloader of the phar is only used when the class is not loadable.

Summary

These files do not replace a proper dependency management. It still is a relic from the old times to bundle third party code and especially when this becomes more public, we all will run into conflicts when one extension requires another incompatible version of the same library. But I consider this OK, to support legacy systems at least for a while.

But if you can, you should start setting up your TYPO3 projects with composer. Mid term the TER will also be replaced with something that speaks composer but not "Extension Manager".

Have fun, stay clean.

[doc-core-insight]

How to configure class loading for extensions in TYPO3 7LTS

tl;dr: In 99,9% of the cases, you do not have to do anything. Every configuration option below is just for clarity and convenience during development or to cover the 0,1% edge case. It is however considered best practice to add any of the following configuration options.

In my last article I gave a lot of background information why we discarded our class loader in TYPO3 7LTS and switched to use the one provided by composer. This post is more a tutorial with a lot more examples. It has a focus on how rather than why and the target group is TYPO3 extension developers.

Minimum setup

In this section I describe what you can do to get better class loading suport in TYPO3 7LTS. Everthing you find here, you can safely do without loosing compatibility with older TYPO3 versions!

Compatiblity mode

If you don't want to, you do not have to do anything! Regarding class loading your extension will work with TYPO3 7LTS anyway in 99,9% of the cases. TYPO3 scans the extension directory for class or interface files and registers every single one automatically to ensure backwards compatibility.

Important: If you add classes during development they will not be found by the class loader unless you update your class information files. If you have classes located e.g. in a tests folder which should not be availble in production, you should consider one of the following ways to configure class loading.

Extensions having one folder with classes

Considering you have an Extbase extension (or an extension where all classes and interfaces reside in a Classes folder) you can simply add the following to your ext_emconf.php file:

'autoload' => array( 'classmap' => array('Classes') ),

The folder Classes will be scanned and every class or interface found will automatically be registered, no matter if the classes use PHP namspaces or not.

Important: If you add classes during development they will not be found by the class loader unless you update your class information

Extensions using namespaces (possible since TYPO3 6.0)

TYPO3 introduced a way to load classes using namespaces, which was later standardized by the PHP framework interoperability group as PSR-4. If your extension has namespaced classes, then you can add the following to your ext_emconf.php file:

'autoload' => array( 'psr-4' => array('Vendor\\ExtName\\' => 'Classes') ),

Here you do not only specify a folder, but also a namespace prefix which all classes in this folder must have. The prefix must end with a backslash!

Note: If you do so, newly added classes during development can be resolved automatically, whithout any further action! There is no need to update your class information files!

Legacy extension which previously used an ext_autoload.php file

As outlined before, it is not necessary any more to specify the name and location of every single class. You can specify folders and TYPO3 scans for the classes in them. So for example to provide class information for tt_news, Rupi could add the following to the ext_emconf.php:

'autoload' => array( 'classmap' => array( 'pi', 'mod1', 'lib', 'class.tx_ttnews_compatibility.php', ) ),

Note: As you can see, it is perfectly fine to add multiple folders or even single files to the class map. However, just to stress it again, if you add classes during development they will not be found by the class loader unless you update your class information files manually.

Advanced setup

This section is a collection of further things you can do to tweak your development experience regarding class loading in TYPO3 7LTS.

Class loading information in composer.json

If your extension has a composer.json file already with a psr-4 or classmap autoload section, you are done. No further action is required. The section in the composer.json file can look like this:

"autoload": { "psr-4": { "Vendor\\ExtName\\": "Classes" }, "classmap": ["Resources/Private/PHP/SomeLib"] },

As you can see, you can combine both classmap and psr-4 sections (which is also possible of course in an ext_emconf.php file.

Fixture classes

The old TYPO3 class loader could automatically resolve classes inside a Tests folder, in case the namespace reflected the folder structure below. In TYPO3 7LTS this is not the case any more, which really is a good thing! If you do not want to use composer to execute your extension tests individually like I described in a previous article, but execute them in the context of your TYPO3 project installation, you can add an autoload-dev section in your ext_emconf.php file:

'autoload-dev' => array( 'psr-4' => array('Vendor\\ExtName\\Tests\\' => 'Tests') ),

Note: Just like composer, TYPO3 ignores an autoload-dev section in a composer.json file. So unlike the regular autoload section, the autoload-dev section is only evaluated if present in the ext_emconf.php file.

General notes

Update class loading information

The class loading information is updated every time you activate or deactivate an extension via extension manager. In the rare case you want to update it manually, you can run the following command from the command line:

typo3/cli_dispatch.phpsh extbase \ extension:dumpclassloadinginformation

Composer mode

It is possible to manage the source code for your TYPO3 project exclusively with composer (which I can highly recommend). In that case, all class loading information provided by extensions in the ext_emconf.php files is completely ignored! However as best practice and for the time being it is recommended to nevertheless provide this information in your extensions, so that it can be used in projects which are not set up using composer.

Enjoy developing with TYPO3 7LTS. It is an awesome release!