#typo3_console

[chriwode]

TYPO3 Projekt in PhpStorm einrichten

Es gibt sicherlich diverse Möglichkeiten ein TYPO3 Projekt in PhpStorm einzurichten. Da es derzeit scheinbar noch kein Best-Practice dazu gibt, möchte ich versuchen meinen Weg zu beschreiben und hoffe darauf Input von vielen anderen Stellen zu bekommen. Mein Weg hat aus meiner Sicht einige Lücken/Schwachpunkte die immer wieder zu verschiedenen Problemen führen.

Um ein wirkliches Best-Practice zu entwickeln werde ich die Beschreibungen in kleinere Beiträge unterteilen. Hierbei sollen die Themen Composer, Surf, typo3_console, Git, PhpStorm, Dokumentation und die Entwicklung einer eigenen Extension in diesem Umfeld besprochen werden.

Zu allen Einzelthemen gibt es bereits genügend Blog- bzw. Gistbeiträge, die ich auch gern nennen möchte. Jedoch ist für mich die Kombination aus allem hier entscheident.

Dieses Blogserie soll sich klar nicht nur an Enwickler richten, sondern den Integrator in den Forderung schieben. Meiner Ansicht nach hat sich das Aufgabengebiet eines Integrators vergrößert, so dass es nicht nur seine Aufgabe ist eine Extension einzurichten.

Teil 1 TYPO3-Projekt mit Composer

Teil 2 Projekt in PhpStorm einbinden

Teil 3 PhpStorm einrichten

Teil 4 Coding Style

Teil 5 Eigene Extension integrieren

Teil 6 Website-Projekt mit Surf deployen

Teil 7 Dokumentieren mit Sphinx und/oder Markdown

Teil 8 Tools in PhpStorm

Gern könnt ihr Themenwünsche in den Kommentaren hinterlassen, so dass diese hier ebenfalls einfliessen können.

[doc-core-insight]

TYPO3 Extension dependencies revisited

tl;dr: Don't try to forsee the future, but add TYPO3 composer dependencies wisely

In my last article "composer.json specification for TYPO3 Extensions" I already specified in a few words, how the require section for extensions should look like.

Let's dissect some of the statements a bit further and add some more explanation and context to it.

The require property MUST be provided and it MUST contain the requirement to TYPO3 with a version constraint.

It should be very easy to understand, that TYPO3 extensions only work with TYPO3 and thus TYPO3 needs to be added as dependency to the composer.json of extensions. But let me elaborate what I recommend as concrete version constraint.

If your extension is compatible with the two latest LTS releases, the version constraint is pretty straightforward: ^6.2 || ^7.6

If your extension relies on a certain bugfix of a certain bugfix version, you can specify that minor release easily: ^6.2.13 || ^7.6.11

Things get a little bit more tricky, if your extension supports current TYPO3 sprint releases. It is important to know that minor versions bumps of TYPO3 sprint releases may contain breaking changes. If your extension is compatible with version 8.0.0, it does not necessarily mean it is compatible with 8.1.0 or 8.2.0.

If you now specify a version constraint of ^8.0.0 or < 8.99.99 at the time when version 8.0.0 is released, then you declare that this exact same version, this exact same code base, will be compatible with all future sprint releases, which in fact you can not, unless you can forsee the future.

If you nevertheless use such a loose version constraint, you are making a bet. If you are lucky, nothing bad happens and your code magically works with a future sprint release. If you are not so lucky, than your incompatible extension version might be installed with the latest TYPO3 sprint release and might be the reason, why a complete TYPO3 installation is broken.

I suggest not to bet, but specify exactly with which sprint release version your extension is compatible: ^7.6 || >=8.3.0 <8.6 would be a perfect version constraint, when 8.5.0 has been released and your extension version is compatible with that version, but needs at least 8.3.0. If 8.6.0 is released and your code still works, just bump the version constraint with one commit and release a new version that is compatible with the latest TYPO3 sprint release. If your code does not work, you made sure, broken code is not installed.

It SHOULD contain at least a requirement to typo3/cms-core. It SHOULD NOT contain a requirement to typo3/cms directly to avoid confusions as extensions cannot be used as root package with TYPO3.

Now what does this mean? The package typo3/cms-core isn't available on Packagist, so why would I want to add a dependency to it? Well, here is the deal: typo3/cms is available on Packagist and declares to be a replacement for typo3/cms-core and all other system extensions of the current TYPO3 core (typo3/cms-backend, typo3/cms-info, …). Currently if you specify typo3/cms-core, your extension composer.json file cannot act as a root package in the sense that you cannot go to the extension directory and simply type composer install. If you do so, you'll get an error message. This is important, because TYPO3 cannot deal with extensions that are not in the typo3conf/ext directory anway.

You may want to install and test multiple TYPO3 version in your CI setup. But even then I find it useful to not specify typo3/cms directly in the extension composer.json and do composer install during CI, because then composer will always only install the latest TYPO3 version your extension is compatible with.

So you would end up doing composer require typo3/cms $TYPO3_VERSION anyway, which would overwrite any previously specified dependency to typo3/cms, but will add a depedency to it alongside letting composer check whether the newly required TYPO3 version is actually compatible with what the extension specifies. Believe me or, this setup caught quite some errors for me and some other people I know. It is not a strict requirement to use typo3/cms-core instead of typo3/cms, but it gives enough benefits to recommend it.

It COULD contain requirements to other individual TYPO3 system extensions, to properly specify which TYPO3 extensions this extension depends on. This is useful especially in combination with TYPO3 Console, but also provides information, which will be useful in the future.

What I mean with that is, that individual core extensions might be split out of the big TYPO3 repository, making it possible to install only these TYPO3 system extensions, that are strictly required for TYPO3 or your project. For this to work properly, extension (or composer package) authors would need to start specifying which exact TYPO3 system extensions their code depends on. I did this for TYPO3 console for example. Doing so now comes with no downside and no side effect. It only enables better compatiblity for the future and allows TYPO3 Console to automatically detect which TYPO3 system extensions need to be active in a project.

Hope this clears some things up. Happy composer.json writing!

[chriwode]

TYPO3-Projekt mit Composer

Teil 1 - Composer ist aus meiner Sicht aus der TYPO3-Welt nicht mehr wegzudenken. Es gibt noch den einen oder anderen der Projekte nicht mit Composer erstellt, was ich mir selbst nicht mehr vorstellen mag.

Kurzer Abriss zu Composer

Für all diejenigen welche noch nicht mit Composer arbeiten sei gesagt, stellt eure Projekte darauf um. Das handling der Projektabhängigkeiten wird hierdurch deutlich vereinfacht. Composer selbst ist ein Paketmanager, durch welchen ihr alle benötigten TYPO3 Extensions installieren könnt.

Anlaufstellen für Composer

Eine der ersten Anlaufstellen ist https://composer.typo3.org/, wo euch rudimentär erklärt wird wie ihr ein Composer basiertes TYPO3 Projekt erstellt. Bei der Vertiefung eurer Composer Kenntnisse hilft euch https://getcomposer.org/.

Darüber hinaus haben sich andere Leute bereits Gedanken darüber gemacht, die Composer-Datei nicht stets neu erstellen zu müssen. Zum einen gehört Helmut Hummel (@helhum) und Stefan Salzmann (@KaffDaddy) dazu, zum anderen habe ich mich auch so meine Gedanken dazu gemacht ;)

Helmut's Composer Varianten

Von Helmut gibt es auf GitHub die TYPO3-Distribution, welche alles enthält um sein eigenes Projekt zu erstellen. Diese Distribution schließt bereits die optionale Nutzung von Surf mit ein. Zudem gibt es auf GistHub von Helmut seine (derzeit) ideale Composer Datei samt Beschreibung.

Stefan's Composer Variante

Auf den Blog von Stefan bin ich erst kürzlich gestoßen, als ich mich mit diesem Best-Practise auseinder gesetzt habe. Ähnlich wie Helmut beschreibt Stefan den Aufbau seiner Composer-Datei mit interessanten Details und wie er diese gleich mit der typo3_console verbindet.

Christian's Composer Variante

Neben oben genannten Möglichkeiten habe ich selbst auch eine Variante auf GitHub veröffentlicht. Mit meiner Variante ist die Nutzung von Surf möglich und auch die typo3_console kommt hier zum Einsatz. Wobei ich ehrlich gestehen muss, dass meine Variante eine ältere Variante von Helmut's TYPO3-Distribution ist, welche jedoch um andere Dinge erweitert wurde.

Auf welchen Weg auch immer ihr eure Composer-Datei erstellt, mittels dem Konsolenbefehl composer install installiert ihr das TYPO3-Projekt mit allen notwendigen Abhängigkeien. Ihr wollt nachträglich eine Extension installieren?, kein Problem! Dazu auf der Konsole lediglich composer require typo3-ter/ext-key tippen und schon wird die Erweiterung installiert.

Welche TYPO3 Extensions über o.g. Befehl installiert werden können, könnt ihr unter https://composer.typo3.org/satis.html herausfinden. Wichtig ist lediglich der Bindestricht im Extensionnamen. Heißt eine Extension beispielsweise cw_best_practice, so muss diese bei Composer als cw-best-practice angegeben werden.

Hinweis von @IchHabRecht (vielen Dank dafür)

Es ist besser eine TYPO3 Extension über den Package Key in die Composer Datei hinzu zufügen. Dazu muss die jeweilige Extension bei Packagist registriert sein. Zudem macht man sich vom TYPO3 Extension Repository (TER) nicht abhängig.

Das bedeutet, wenn ihr beispielsweise die Extension "News" von Georg Ringer nutzen möchtet, reicht ein composer require georgringer/news und die Extension wird in der aktuellsten Version für die euere TYPO3 Installation installiert.

Ist man mit Composer zwischenzeitlich warm geworden und mag nicht mehr ohne arbeiten, dann ist der Blogbeitrag von Marcus Schwemmer sehr hilfreich, in welchem er das Patchen von Erweiterungen bzw. auch vom TYPO3 Core beschreibt. Ich konnte dies bereits auch 2x verwenden und es ist ein Traum wie einfach und schnell das geht.

Wenn jemand weitere interessante Links zu diesem Thema hat, bitte diese gern als Kommentar hinterlassen, so dass diese hier mit aufgeführt werden können.

[doc-core-insight]

About the beauty and power of TYPO3 Console

tl;dr: I'm often asked why I created TYPO3 Console and how it compares to other cli tools in the TYPO3 universe. So I decided to write a few lines about why and how I created a tool I enjoy using.

Suffering from the status quo

While working on a deployment solution for TYPO3 CMS it become clear to me that during deployment I would need a certain set of commands to be executed from command line. In the beginning I happily used the great extension coreapi. During that time I also had to use other command line tools from the TYPO3 core. Calling commands looked like this:

# Execute scheduler ./typo3/cli_dispatch.phpsh scheduler # Update translations ./typo3/cli_dispatch.phpsh extbase language:update # Check reference index ./typo3/cli_dispatch.phpsh lowlevel_refindex -c # Update database schema ./typo3/cli_dispatch.phpsh extbase databaseapi:databasecompare 2,3,4

Do you see any consistency? I didn't. This is a small sample of old school TYPO3 cli scripts, nicer Extbase (TYPO3 Flow like) command controllers. Every single one of them has a different way to be called, to get help texts, to list available commands and how arguments are named and used. On top of this confusion, there is the need to add backend user records for many of them, but not for all. Inconsistency all over the place.

Kickstarting change

I just couldn't stand that situation and started creating a tool that can execute commands based on command controllers with the intention to use it together with coreapi (which would provide the commands).

I very quickly had a first version of a typo3cms command running and could execute command controller commands like that:

# Update translations ./typo3cms language:update # Flush caches ./typo3cms cacheapi:clearallcaches # Update database schema ./typo3cms databaseapi:databasecompare 2,3,4

I felt much better. However there were still some TYPO3 core commands left that weren't command controllers so I still had to deal with the old dispatcher script which I found to be a bummer.

Understandable and consistent interface

Thus I started to recreate most of the TYPO3 cli script functionality as command controllers:

# Lock backend ./typo3cms backend:lock # Check reference index ./typo3cms cleanup:updatereferenceindex --dry-run # Run scheduler ./typo3cms scheduler:run

Awesome. Consistent, cleaned up code, meaningful and consistent command and argument names. Meaningful and consistent names? I really started to desire this for all commands including the ones from coreapi, which suddenly didn't look nice enough to me any more.

Why would I need to flush caches with ./typo3cms cacheapi:clearallcaches? And what the heck does the argument 2,3,4 for the database compare command actually mean? Wouldn't it be much nicer to type and read ./typo3cms cache:flush? I thought it would. So I created a cache command controller to get exactly such a command.

Speaking of caches. Since I was now busy creating a flush cache command, I thought it would also be nice to be able to reliably clear caches from command line (even if invalid information got cached e.g. because of a broken extension during development, which causes TYPO3 quit due to a fatal error). Since TYPO3 6.2 this is possible in the install tool because it uses a more robust bootstrap than the frontend, backend and frankly the command line interface.

Flexible bootstrap to create powerful commands

Since I not only wanted a nice but also a reliable and robust flush cache command, I had to create a command line bootstrap for TYPO3, that is more robust than it was before. Since I also needed this bootstrap to be as compatible as possible with existing command controllers, I in addition needed to boot a full command line bootstrap.

So I spent an irresponsible amount of time to more or less re-write the bootstrap to be granular, flexible and robust. In the end I could boot to a low "run level", flush core caches without interference of (broken) extensions and then (in the same request) boot into a higher run level, so that all caches including the once from extensions could be cleared.

To perfect the cache command collection I added commands to flush by tag and group and also a command to warm up essential core caches (e.g. after having them flushed beforehand during deployment).

Having that in place I could create commands that worked without a database or the regular TYPO3 folder structure being present. Since it came in handy for me, I created a command to set up a blank TYPO3 instance directly from the command line instead of using the install tool in the browser:

./typo3cms install:setup --non-interactive \ --database-user-name="root" \ --database-host-name="localhost" \ --database-port="3306" \ --database-name="travis_test" \ --admin-user-name="admin" \ --admin-password="password" \ --site-name="Test Instance"

One more thing

While I was working on my baby, the folks from TYPO3 Flow nicely integrated Symfony console into the command line interface. Since Extbase command controllers basically were a back port from Flow, I could quickly adapt and integrate this change from them into typo3_console.

With this change interactive user input on the command line became also possible for typo3_console commands. When calling a command e.g. ./typo3cms install:setup you'll be asked for all required arguments if they aren't specified beforehand

I also integrated a progress bar, that was available since then, into the reference index update command, to be able to better estimate how long this long running intensive task may take.

Conclusion

I pointed out what I like most from the tool I created. And there are so much more nice details, which I haven't mentioned. If you think you can also benefit from the love that went into it, I would be happy if you'd try it out.

Have fun!