How to handle volumes?

[slashrsm]

Dockerfile defines default volume:
VOLUME /var/www/html

This was totally ok before we enabled configurable docroots, but now it can happen that we create this default volume and never use it and/or don't create volume for the actual docroot. Same issue exists in php images.

Any idea how to solve that? Should we remove assumption about the volume and let users decide on that? Would it be possible to create it dynamically based on the docroot configuration?

[zaporylie]

TL;DR
Default VOLUMEs are handy when it comes to multi-container projects. All php images defines same path ("project" path - /var/www/html) which simplify data sharing between containers with almost zero configuration.

When you instantiate image (create container), VOLUME instruction in Dockerfile tells docker engine which directory inside the container should be defined as a volume. Volumes can be shared between containers (with --volumes-from param to docker run or volumes_from in docker-compose.yml). See https://github.com/drupal-docker/compose/blob/master/examples/04-networks/docker-compose.yml#L22 for reference.

For PHP projects, like Drupal, it makes perfect sense to have whole project inside of volume, not only DOCROOT. Let's take drupal-project with three services - nginx <-> php:fpm <-> drush - as an example.

Project structure looks more or less like that:
/var/ww/html/drush (php code)
/var/ww/html/web (DOCROOT, php code)
/var/ww/html/private (static private files)
/var/ww/html/vendor (php code)

Let's say - hypothetically - that volume points to DOCROOT (/var/ww/html/web) and is mounted (or copied) to php:fpm container. That way nginx container could not access private files (those file are not available inside of volume). Also drush will fail as it doesn't have access to config in /drush subfolder and packages from /vendor.

(to be continued...)

[iBobik]

@zaporylie In Docker it is best practice to have own application copied into app container while build, so it works nicely with Docker building and deployment solutions.

How to share this codebase with nginx container?

[zaporylie]

@iBobik tricky question, but as you said building/compiling/copying application into image is Docker best practice. Basically to share codebase across app (nginx/php-fpm/php-cli/etc.) you MUST declare files copied into image as a volume. This is the only way to build artifact-based multicontainer app, and AFAIK it can be done only this way:

1. In Dockerfile use ADD/COPY to add project files to image.
2. Declare path you copied files to as VOLUME (in Dockerfile) or on container run (-v).

What's important here is the declaration chain. First you have to copy files over THEN declare it as a volume. When you create new project (i.e. with docker-compose up) docker will figure out that this volume, let's say, /var/www/html hasn't been created so far and it will use default content for it stored in image (first step above).

It gets job done but only to the point when you're gonna release new version of project image (create new artifact). Already declared volumes take precedence over newly created volumes so your codebase will remain the same (updates to volume will be discarded). This is by design as you don't want to loose user data for existing application (i.e. /var/www/html/web/sites/default/files), don't you? 😏

Unfortunately this is why Docker doesn't fit so well to PHP apps. There's a workaround tho (however I didn't have enough time to explore it properly - using simplified version of it and it works fine). You can build/compile/copy your application into custom-declared directory somewhere in image's filesystem (i.e. /app), add service based on this image to docker-compose.yml (i.e. build) and make all other services re-use volume (volumes_from) declared by this service (/var/www/html). The only command build service will execute is to sync changes to project files between /app and /var/www/html using rsync or similar tool. After that it will exit and become a soft-dependency for all other services (php, nginx, etc.). To upgrade project to new version you would have to pull new version of build from docker hub (or similar) and run something like docker-compose up build (or more likely docker-compose up deploy to run deploy service which, in addition to re-build, executes deploy script, i.e. drush updb && drush cim && drush cr).

I hope I made it clear but if you I feel like I did to many shortcuts on the way or you want me to explain some parts of this complex topic more - please share your thoughts and doubts.

[iBobik]

Seems very interesting. The build service should have just one simple bash script for sync and Drupal project, shouldn’t it? And this script should just copy everything except sites/default/files into volume and then exit (so container is needed to run only once after deploy and then stop)? Jan Pobořil 2016-12-12 21:55 GMT+01:00 Jakub Piasecki <notifications@github.com>:

…

@iBobik <https://github.com/iBobik> tricky question, but as you said building/compiling/copying application into image is Docker best practice. Basically to share codebase across app (nginx/php-fpm/php-cli/etc.) you MUST declare files copied into image as a volume. This is the only way to build artifact-based multicontainer app, and AFAIK it can be done only this way: 1. In Dockerfile use ADD/COPY to add project files to image. 2. Declare path you copied files to as VOLUME (in Dockerfile) or on container run (-v). What's important here is the declaration chain. First you have to copy files over THEN declare it as a volume. When you create new project (i.e. with docker-compose up) docker will figure out that this volume, let's say, /var/www/html hasn't been created so far and it will use default content for it stored in image (first step above). It gets job done but only to the point when you're gonna release new version of project image (create new artifact). Already declared volumes take precedence over newly created volumes so your codebase will remain the same (updates to volume will be discarded). This is by design as you don't want to loose user data for existing application (i.e. /var/www/html/web/sites/default/files), don't you? 😏 Unfortunately this is why Docker doesn't fit so well to PHP apps. There's a workaround tho (however I didn't have enough time to explore it properly - using simplified version of it and it works fine). You can build/compile/copy your application into custom-declared directory somewhere in image's filesystem (i.e. /app), add service based on this image to docker-compose.yml (i.e. build) and make all other services re-use volume (volumes_from) declared by this service (/var/www/html). The only command build service will execute is to sync changes to project files between /app and /var/www/html using rsync or similar tool. After that it will exit and become a soft-dependency for all other services (php, nginx, etc.). To upgrade project to new version you would have to pull new version of build from docker hub (or similar) and run something like docker-compose up build (or more likely docker-compose up deploy to run deploy service which, in addition to re-build, executes deploy script, i.e. drush updb && drush cim && drush cr). I hope I made it clear but if you I feel like I did to many shortcuts on the way or you want me to explain some parts of this complex topic more - please share your thoughts and doubts. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#14 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAlfWBiQ5b7aNro5KNFjm2G3g9m3iGnGks5rHbTTgaJpZM4KcRdk> .

[zaporylie]

To be precise - only once before running deploy script. You must have files in place before you run database update & config import & cache clear.
And sync rather than copy as you want to delete deprecated dependencies (i.e. files from vendor which do not belong to project anymore)

[iBobik]

How about running deploy scripts (drush updb, fra, …)? Is it good idea to run it in the php container triggered by a HTTP ping? Or there is some better way?

[zaporylie]

Well, usually it will be triggered by some sort of webhook - from travis, github, gitlab, docker cloud etc. It depends how is your project organized. I believe the best approach in multicontainer artifact-image driven application is to react on image push to Docker Registry (Hub, gitlab or something self-hosted). I tested it with Docker Cloud (they support triggering autorebuild on image push) and it actually works quite well. I have plan to look at it more (for tools like Docker Cloud, GitLab support for Docker) but it requires spare time I don't have right now. At work we use Jenkins to trigger deploy service.