FusionForge

Difference between revisions of "Docker"

From FusionForge Wiki
Jump to: navigation, search
m
 
(20 intermediate revisions by 5 users not shown)
Line 1: Line 1:
'''Creating a FusionForge development/tests environment using Docker containers'''
+
[[Category: Development]]
  
This HOWTO is meant to illustrate how Docker lightweight containers can be used to set-up a development and test environment for FusionForge, '''when it is complete'''.
+
This howto takes as hypothesis that you are aware of docker and how to use docker.<br/>
 +
The docker container is not for production usage but for demo or testing.
 +
==Get the docker files==
 +
The docker files are published in the main fusionforge scm repository: [https://scm.fusionforge.org/anonscm/gitweb?p=fusionforge/fusionforge.git;a=tree;f=docker;h=f51c9423c524d7d744cc0bf0ec64fd13cf5012b4;hb=HEAD here]<br/>
 +
Download each file into a working directory.
 +
==Build the image==
 +
To build the image just use:
 +
# cd <your_working_directory>
 +
# docker build -t <your_image_name> .
 +
The build takes about 15 minutes.
 +
==Get the image from docker hub==
 +
The FusionForge image is published on docker hub [https://hub.docker.com/r/fusionforge/fusionforge/ repository]. You can pull the image from there. The new image is published almost every month for the master branch.
  
'''This document is work in progress and is probably misleading in the current state, sorry.'''
+
==Run the container==
 +
FusionForge requires a valid FQDN and the container exposes 4 ports:
 +
* 21 for ftp access
 +
* 22 for ssh access
 +
* 80 & 443 for http & https access
 +
You need to map these ports locally.<br/>
 +
You can use some variables to set the internal configuration of FusionForge:
 +
* PORT_SSH will configure the ssh port for plugins such as scmgit or scmsvn
 +
* PORT_HTTP will configure the internal URL to build the different http link
 +
* PORT_HTTPS ditto PORT_HTTP but for https
 +
FusionForge includes vsftpd server with chroot mechanism. This mechanism uses the mount command to bind internal folders. Therefore you need the CAP_SYS_ADMIN capability to get ftp features working properly. Use
 +
Standard running command:
 +
# docker run --cap-add=SYS_ADMIN -h <your_valid_fqdn> -e PORT_SSH=2222 -p 2222:22 -p 21:21 -p 80:80 -p 443:443 -d <your_image_name>
  
The principle is to prepare an image of a Debian distribution containing an installation of PostgreSQL and Apache2 which can be used to create a container where FusionForge will be installed.
+
==Set admin password==
 
+
By default FusionForge installation comes with one user: admin but no password is set to this user. You need to set the password of admin user. Use the following command to connect to the container:
A similar setup can probably be achieved using other distribution packages.
+
  # docker exec -it <container_id> /bin/bash
 
+
Then set the admin user password
== Installing Docker on Debian testing/sid ==
+
  # forge_set_password admin <your_password>
 
+
==Use the container==
Install the package (note that the 'docker' command is renamed in 'docker.io' to avoid conflict with a pre-existing package):
+
Open your browser and go to <your_valid_fqdn><br/>
apt-get install docker.io
+
You can connect as user with the login/password: admin/<your_password><br/>
ln -s /usr/bin/docker.io /usr/local/bin/docker
+
Enjoy!
 
 
If you already played with LXC/cgroups, make sure you '''remove''' your /sys/fs/cgroup entry from /etc/fstab, and let the cgroupfs-mount package take care of it, otherwise you'll get clear and informative errors such as:
 
[error] client.go:2329 Error getting size: bad file descriptor
 
 
 
 
 
== Creating an image containing a Debian sid + PostgreSQL + Apache 2 ==
 
 
 
We will provide example DockerFiles for (stacked over each-other) :
 
* a Debian sid container running the OpenSSH server
 
* an image containing an Apache 2 server, based on a Debian sid + SSH one;
 
* an image containing a PostgreSQL server, based on an Apache one;
 
* an image containing fusionforge-minimal, based on the PostgreSQL one;
 
 
 
=== Debian sid/unstable container running the OpenSSH server ===
 
 
 
You may use one of the methods described below:
 
 
 
==== Use a Debian variant of baseimage-docker ====
 
 
 
See : https://github.com/olberger/baseimage-docker/tree/debian
 
 
 
Note we changed the <tt>ENTRYPOINT ["/usr/sbin/runsvdir-start"]</tt> last line of the Dockerfiles below to <tt>CMD ["/sbin/my_init"]</tt>.
 
 
 
git clone https://github.com/olberger/baseimage-docker
 
cd baseimage-docker/
 
sudo make
 
sudo docker tag phusion/baseimage:0.9.8 myfusionforge/sid_ssh
 
 
 
Then you can proceed with [[#Apache_2_server_container]] below.
 
 
 
==== Create an image based on the "official" Debian/sid image, to add OpenSSH server ====
 
 
 
Adapt this to your needs (changing the MAINTAINER declaration, for instance).
 
 
 
This will add an SSH server (started through runit) over a base Debian sid image fetched from the official docker.io repository. You may prefer to generate one for yourself: see https://wiki.debian.org/Cloud/CreateDockerImage.
 
 
 
<nowiki>#
 
# Simple dockerfile for a Debian sid/unstable + ssh.
 
#
 
# Build it like so:
 
#
 
#  root@host~# docker build -t=myfusionforge/sid_ssh - < Dockerfile.sid_ssh
 
#
 
# Launch the generated image like so:
 
#
 
#  root@host~# docker run -t -d -p 2222:22 myfusionforge/sid_ssh
 
#
 
# Connect like so, with the root password being "whatever123".
 
#
 
#  $ ssh -p 2222 root@localhost
 
#
 
# Adapted from http://www.debian-administration.org/article/698/Automating_the_creation_of_docker_images by Olivier Berger
 
#
 
 
 
 
 
 
 
#
 
#  From this base-image / starting-point
 
#
 
#FROM myfusionforge/base:sid
 
FROM debian:sid
 
 
 
#
 
#  Authorship
 
#
 
MAINTAINER olivier.berger@telecom-sudparis.eu
 
 
 
ENV DEBIAN_FRONTEND noninteractive
 
ENV DEBIAN_PRIORITY critical
 
ENV DEBCONF_NOWARNINGS yes
 
 
 
#
 
# Update apt
 
#
 
RUN apt-get update -q -q
 
RUN apt-get upgrade --yes --force-yes
 
 
 
#
 
# Install utilities
 
#
 
RUN apt-get install less sudo screen --yes --force-yes
 
 
 
#
 
# Install runit
 
#
 
RUN apt-get install runit --yes --force-yes
 
 
 
#
 
# Install SSH
 
#
 
RUN apt-get install openssh-server openssh-client --yes --force-yes
 
 
 
#
 
# Setup a root password; simple enough to remember, but hard enough that
 
# it won't be cracked immediately.  (ha!)
 
#
 
RUN echo "root:whatever123" | chpasswd
 
 
 
#
 
# Expose the SSH port
 
#
 
EXPOSE 22
 
 
 
#
 
#  Now make sure that runit will launch SSHD, via runit.
 
#
 
#  NOTE: Remember runit will launch /etc/service/sshd/run
 
#
 
RUN mkdir /etc/service/sshd
 
RUN /bin/echo -e '#!/bin/sh' > /etc/service/sshd/run
 
RUN /bin/echo -e 'if [ ! -d /var/run/sshd ]; then mkdir -p /var/run/sshd ; fi' >> /etc/service/sshd/run
 
RUN /bin/echo -e 'exec /usr/sbin/sshd -D' >> /etc/service/sshd/run
 
 
 
#
 
#  Make sure our run-script is executable.
 
#
 
RUN chown root.root /etc/service/sshd/run
 
RUN chmod 755 /etc/service/sshd/run
 
 
 
 
 
#
 
# Finally launch runit.
 
#
 
ENTRYPOINT ["/usr/sbin/runsvdir-start"]
 
</nowiki>
 
 
 
=== Apache 2 server container ===
 
 
 
This will reuse the <tt>myfusionforge/sid_ssh</tt> image to add Apache 2 to it, also run through runit.
 
 
 
Note that to restart the apache2 server, you'll need to do <tt>apache2ctl stop</tt> (and runit will restart it for you immediatly, unless you first do a <tt>sv down apache2</tt>).
 
 
 
Copy this to Dockerfile.sid_apache and follow the instructions in the comments:
 
  <nowiki>
 
#
 
# Simple dockerfile for a Debian sid/unstable image running an Apache 2 server through runit.
 
#
 
# Build it like so:
 
#
 
#  root@host~# docker build -t=myfusionforge/sid_apache - < Dockerfile.sid_apache
 
#
 
# Launch the generated image like so:
 
#
 
#  root@host~# docker run -t -d -p 2222:22 -p 8080:80 -p 4443:443 myfusionforge/sid_apache
 
#
 
# Connect like so, with the root password being "whatever123".
 
#
 
#  $ ssh -p 2222 root@localhost
 
#
 
# Adapted from http://www.debian-administration.org/article/698/Automating_the_creation_of_docker_images by Olivier Berger
 
#
 
 
 
 
 
 
 
#
 
#  From this base-image / starting-point
 
#
 
FROM myfusionforge/sid_ssh
 
 
 
#
 
#  Authorship
 
#
 
MAINTAINER olivier.berger@telecom-sudparis.eu
 
 
 
#ENV DEBIAN_FRONTEND noninteractive
 
#ENV DEBIAN_PRIORITY critical
 
#ENV DEBCONF_NOWARNINGS yes
 
 
 
#
 
# Update apt
 
#
 
RUN apt-get update -q -q
 
RUN apt-get upgrade --yes --force-yes
 
 
 
#
 
# Install Apache
 
#
 
RUN apt-get install apache2 --yes --force-yes
 
 
 
#
 
# Expose the Apache HTTP/HTTPS ports
 
#
 
 
 
EXPOSE 80
 
EXPOSE 443
 
 
 
#
 
#  Now make sure that runit will launch Apache2
 
#
 
#  NOTE: stopping it will be done with : sv down apache2; apache2ctl stop
 
#
 
RUN /bin/echo -e "export APACHE_ARGUMENTS='-DNO_DETACH'" >> /etc/apache2/envvars
 
 
 
RUN mkdir /etc/service/apache2
 
RUN /bin/echo -e '#!/bin/sh' > /etc/service/apache2/run
 
RUN /bin/echo -e 'exec /usr/sbin/apache2ctl start' >> /etc/service/apache2/run
 
 
 
#
 
#  Make sure our run-script is executable.
 
#
 
RUN chown root.root /etc/service/apache2/run
 
RUN chmod 755 /etc/service/apache2/run
 
 
 
#
 
# Finally launch runit.
 
#
 
ENTRYPOINT ["/usr/sbin/runsvdir-start"]
 
</nowiki>
 
 
 
=== PostgreSQL container ===
 
 
 
This time, we add PostgreSQL in addition to Apache 2 and SSH.
 
 
 
Copy this to Dockerfile.sid_postgresql and follow the instructions in the comments:
 
<nowiki>
 
#
 
# Simple dockerfile for a Debian sid/unstable image running Apache 2 and PostgreSQL servers through runit.
 
#
 
# Build it like so:
 
#
 
#  root@host~# docker build -t=myfusionforge/sid_postgresql - < Dockerfile.sid_postgresql
 
#
 
# Launch the generated image like so:
 
#
 
#  root@host~# docker run -t -d -p 2222:22 -p 8080:80 -p 4443:443 myfusionforge/sid_postgresql
 
#
 
# Connect like so, with the root password being "whatever123".
 
#
 
#  $ ssh -p 2222 root@localhost
 
#
 
# Adapted from http://www.debian-administration.org/article/698/Automating_the_creation_of_docker_images by Olivier Berger
 
#
 
 
 
 
 
 
 
#
 
#  From this base-image / starting-point
 
#
 
FROM myfusionforge/sid_apache
 
 
 
#
 
#  Authorship
 
#
 
MAINTAINER olivier.berger@telecom-sudparis.eu
 
 
 
#ENV DEBIAN_FRONTEND noninteractive
 
#ENV DEBIAN_PRIORITY critical
 
#ENV DEBCONF_NOWARNINGS yes
 
 
 
#
 
# Update apt
 
#
 
RUN apt-get update -q -q
 
RUN apt-get upgrade --yes --force-yes
 
 
 
#
 
# Install Postgresql
 
#
 
 
 
RUN apt-get install postgresql --yes --force-yes
 
 
 
 
 
EXPOSE 80
 
EXPOSE 443
 
 
 
 
 
#
 
#  Now make sure that runit will launch PostgreSQL
 
#
 
#  NOTE: stopping it will be done with : sv down postgresql
 
#
 
RUN mkdir /etc/service/postgresql
 
RUN /bin/echo -e '#!/bin/sh' > /etc/service/postgresql/run
 
RUN /bin/echo 'cd /var/lib/postgresql' >> /etc/service/postgresql/run
 
RUN /bin/echo 'exec chpst -u postgres:postgres:ssl-cert /usr/lib/postgresql/9.3/bin/postgres -D /var/lib/postgresql/9.3/main -c config_file=/etc/postgresql/9.3/main/postgresql.conf' >> /etc/service/postgresql/run
 
 
 
 
 
#
 
#  Make sure our run-script is executable.
 
#
 
RUN chown root.root /etc/service/postgresql/run
 
RUN chmod 755 /etc/service/postgresql/run
 
 
 
#
 
# Finally launch runit.
 
#
 
ENTRYPOINT ["/usr/sbin/runsvdir-start"]
 
</nowiki>
 
 
 
== Dockerfile for fusionforge-minimal ==
 
 
 
Copy this to Dockerfile.sid_fusionforge-minimal and follow the instructions in the comments:
 
<nowiki>
 
#
 
# Simple dockerfile for Debian sid/unstable image running a fusionforge-minimal installation.
 
#
 
# Build it like so:
 
#
 
#  root@host~# docker build -t=myfusionforge/sid_fusionforge-minimal - < Dockerfile.sid_fusionforge-minimal
 
#
 
# Launch the generated image like so:
 
#
 
#  root@host~# docker run -t -d -p 2222:22 -p 8080:80 -p 4443:443 myfusionforge/sid_fusionforge-minimal
 
#
 
# Connect like so, with the root password being "whatever123".
 
#
 
#  $ ssh -p 2222 root@localhost
 
#
 
# Adapted from http://www.debian-administration.org/article/698/Automating_the_creation_of_docker_images by Olivier Berger
 
#
 
 
 
 
 
 
 
#
 
# From this base-image / starting-point
 
#
 
FROM myfusionforge/sid_postgresql
 
 
 
#
 
#  Authorship
 
#
 
MAINTAINER olivier.berger@telecom-sudparis.eu
 
 
 
#ENV DEBIAN_FRONTEND noninteractive
 
#ENV DEBIAN_PRIORITY critical
 
#ENV DEBCONF_NOWARNINGS yes
 
 
 
RUN echo "deb http://http.debian.net/debian sid main" > /etc/apt/sources.list
 
 
 
#
 
# Update apt
 
#
 
RUN apt-get update -q -q
 
RUN apt-get upgrade --yes --force-yes
 
 
 
# Fusionforge installation involves modifying files shipped by other
 
# packages, and we need to choose the default UCF option which erases
 
# previous configuration by installing maintainer's version.
 
ENV UCF_FORCE_CONFFNEW yes
 
 
 
# Start PostgreSQL whose database is needed for fusionforge installation, right before installing
 
RUN /etc/init.d/postgresql start ; apt install fusionforge-minimal --yes --force-yes
 
 
 
#
 
# Finally launch runit.
 
#
 
ENTRYPOINT ["/usr/sbin/runsvdir-start"]
 
</nowiki>
 
 
 
=== Creating a container ===
 
 
 
Use the dockerfile to create a container with something like :
 
<code>
 
: $ docker run -t -d -p 2222:22 -p 8080:80 -p 4443:443 myfusionforge/sid_fusionforge-minimal
 
</code>
 
 
 
''Voilà'', FusionForge should then be installed and available at: <tt>https://localhost:4443/</tt>.
 
 
 
=== Forge configuration ===
 
 
 
Modify the host by editing <tt>/etc/gforge/config.ini.d/zzzz-local.ini</tt> to add :
 
 
 
<nowiki>
 
[core]
 
web_host = localhost
 
https_port = 4443
 
</nowiki>
 
 
 
The <tt>admin</tt> user's password can then be reset using:
 
 
 
<code>
 
: root@2eebbd24093c:/usr/share/gforge# bin/forge_set_password admin whatever123
 
</code>
 
 
 
allowing you to login at <tt>https://localhost:4443/account/login.php?return_to=/</tt> with user <tt>admin</tt> and password <tt>whatever123</tt>.
 
 
 
 
 
=== Saving the installation in a Docker image ===
 
 
 
At any time, the container which contains an installation of FusionForge can be saved as an image with the following (where 5.2.3-1 is the tag set to the value of the Debian package version in unstable):
 
 
 
<code>
 
: $ docker commit 9060bc9d0b94 myfusionforge/ff-minimal:5.2.3-1
 
</code>
 
 
 
Note: it may be advisable to first stop apache2 and postgresql before saving the image... TO BE TESTED
 
 
 
A new container can then be created from that image, using something like:
 
 
 
<code>
 
: $ docker run -t -d -p 2222:22 -p 8080:80 -p 4443:443 myfusionforge/ff-minimal:5.2.3-1 /usr/sbin/runsvdir-start
 
</code>
 
 
 
=== Sharing files between host and guest ===
 
 
 
To ease the process of testing changes on the guest, by directly modifying files from the host, it may be useful to share the contents of <tt>/usr/share/gforge</tt> between the host and guest.
 
 
 
This can be done, before installing fusionforge, by launching the PostgreSQL + Apache 2 container with something like :
 
<code>
 
: $ docker run -v ~/docker/usr-share-gforge:/usr/share/gforge -t -d -p 2222:22 -p 8080:80 -p 4443:443 myfusionforge/sid_postgresql
 
</code>
 
 
 
This will install all fusionforge files so that they can be edited (sudo helps) directly from the host in <tt>~/docker/usr-share-gforge</tt>
 
 
 
== Troubleshootings / FIXME ==
 
 
 
Things to fix:
 
* need to regen SSH keys, workaround:
 
sudo chroot /var/lib/docker/containers/6cfgnagnagna/root/
 
dpkg-reconfigure openssh-server
 
* gforge-db-postgresql failing during SQL initial replay
 
TODO
 

Latest revision as of 20:29, 25 September 2017


This howto takes as hypothesis that you are aware of docker and how to use docker.
The docker container is not for production usage but for demo or testing.

Get the docker files

The docker files are published in the main fusionforge scm repository: here
Download each file into a working directory.

Build the image

To build the image just use:

# cd <your_working_directory>
# docker build -t <your_image_name> .

The build takes about 15 minutes.

Get the image from docker hub

The FusionForge image is published on docker hub repository. You can pull the image from there. The new image is published almost every month for the master branch.

Run the container

FusionForge requires a valid FQDN and the container exposes 4 ports:

  • 21 for ftp access
  • 22 for ssh access
  • 80 & 443 for http & https access

You need to map these ports locally.
You can use some variables to set the internal configuration of FusionForge:

  • PORT_SSH will configure the ssh port for plugins such as scmgit or scmsvn
  • PORT_HTTP will configure the internal URL to build the different http link
  • PORT_HTTPS ditto PORT_HTTP but for https

FusionForge includes vsftpd server with chroot mechanism. This mechanism uses the mount command to bind internal folders. Therefore you need the CAP_SYS_ADMIN capability to get ftp features working properly. Use Standard running command:

# docker run --cap-add=SYS_ADMIN -h <your_valid_fqdn> -e PORT_SSH=2222 -p 2222:22 -p 21:21 -p 80:80 -p 443:443 -d <your_image_name>

Set admin password

By default FusionForge installation comes with one user: admin but no password is set to this user. You need to set the password of admin user. Use the following command to connect to the container:

# docker exec -it <container_id> /bin/bash

Then set the admin user password

# forge_set_password admin <your_password>

Use the container

Open your browser and go to <your_valid_fqdn>
You can connect as user with the login/password: admin/<your_password>
Enjoy!