FusionForge

Difference between revisions of "Virtual machine development environment"

From FusionForge Wiki
Jump to: navigation, search
(Precisions on running the test suite)
(Split tips in subsection to improve readability)
Line 4: Line 4:
  
 
See : http://lists.fusionforge.org/pipermail/fusionforge-general/2010-November/001245.html for more details
 
See : http://lists.fusionforge.org/pipermail/fusionforge-general/2010-November/001245.html for more details
 +
 +
== Presentation ==
  
 
Here's an excerpt from the enclosed <tt>/root/Desktop/README.html</tt>:
 
Here's an excerpt from the enclosed <tt>/root/Desktop/README.html</tt>:
Line 13: Line 15:
 
with a checkout of the FusionForge code and a few helper scripts. Those scripts are in the '''/root/scripts''' directory:
 
with a checkout of the FusionForge code and a few helper scripts. Those scripts are in the '''/root/scripts''' directory:
  
  • '''update.sh''' will update the sources from FusionForge's Subversion repository, and also the currently installed packages.
+
* '''update.sh''' will update the sources from FusionForge's Subversion repository, and also the currently installed packages.
  • '''build.sh''' builds FusionForge packages from the sources, and stores them in the package repository at /root/debian-repository. (rem Cbac starting from a debboostraped vm, needs to add dupload, bzr-svn and svn).
+
* '''build.sh''' builds FusionForge packages from the sources, and stores them in the package repository at /root/debian-repository. (rem Cbac starting from a debboostraped vm, needs to add dupload, bzr-svn and svn).
  • '''install.sh''' installs those FusionForge packages. This should end up with an up-to-date FusionForge running on http://forge.local/.
+
* '''install.sh''' installs those FusionForge packages. This should end up with an up-to-date FusionForge running on http://forge.local/.
  • '''start-selenium.sh''' gets and runs Selenium, which is needed for the testsuite.
+
* '''start-selenium.sh''' gets and runs Selenium, which is needed for the testsuite.
  • '''run-testsuite.sh''' actually starts the testsuite.
+
* '''run-testsuite.sh''' actually starts the testsuite.
  • '''install-gui.sh''' sets up an X11 graphical environment.
+
* '''install-gui.sh''' sets up an X11 graphical environment.
  
 
The actual source code is stored under /root/fusionforge-trunk, so that's where you should do the changes you want to test.
 
The actual source code is stored under /root/fusionforge-trunk, so that's where you should do the changes you want to test.
Line 42: Line 44:
 
  bzr checkout https://fusionforge.org/anonscm/bzr/deb-packaging/unstable fusionforge-trunk
 
  bzr checkout https://fusionforge.org/anonscm/bzr/deb-packaging/unstable fusionforge-trunk
  
You may reuse the pre-configured VM image with Virtualbox (or any other VM system, converting the image to the appropriate format), which is the easiest path, or setup a development environment on a Debian Wheezy system you installed on your own, by using the <tt>VM-scripts/configure-scripts.sh</tt> script (see below).
+
You may use this pre-configured VM image with Virtualbox (or convert it to another VM system), or install a development environment on an existing Debian Wheezy system using the <tt>VM-scripts/configure-scripts.sh</tt> script (see below).
  
 
== Tips ==
 
== Tips ==
  
* To reset the admin password, use: <tt>/usr/share/gforge/bin/forge_set_password admin</tt>
+
=== To reset the admin password ===
  
* Converting from Virtualbox image format to qemu QCOW2: convert to raw format first
+
Use: <tt>/usr/share/gforge/bin/forge_set_password admin</tt>
  $ VBoxManage clonehd -format RAW FFSandbox-disk1.vmdk ffsandbox.raw
+
 
 +
=== Converting from Virtualbox image format to qemu QCOW2 ===
 +
 
 +
Converting from Virtualbox image format to qemu QCOW2: convert to raw format first
 +
  VBoxManage clonehd -format RAW FFSandbox-disk1.vmdk ffsandbox.raw # ~4GB
  
 
Then convert to QCOW2 format:
 
Then convert to QCOW2 format:
  $ qemu-img convert -f raw -O qcow2 ffsandbox.raw ffsandbox.qcow2
+
  qemu-img convert -f raw -O qcow2 ffsandbox.raw ffsandbox.qcow2
  
* Running inside qemu:
+
=== Running inside qemu ===
** Create a VM by importing the created qcow2 disk image, with <tt>virt-manager</tt>.
 
** You may need to set manually the driver type to 'qcow2' in "disk" device section in the VM xml file (in /etc/libvirt/qemu/). In this case you need to redefine the VM xml file by: `sudo virsh -c qemu:///system define /etc/libvirt/qemu/ffsandbox.xml` then start the VM.
 
** You need to reconfigure network interfaces in <tt>/etc/network/interfaces</tt> afterwards for replicating <tt>eth0</tt> config stanza for <tt>eth1</tt> for instance. You can find which device to enable by running <tt>ifconfig -a</tt>.
 
** You may need to check the <tt>web_host</tt> setting in <tt>/etc/fusionforge/config.ini.d/debian-install.ini</tt> to adjust the forge's URLs for redirections.
 
  
* It may not harm using <tt>aptitude</tt> inside the VM to upgrade at first launch, eventually taking this opportunity to configure the proper keymap and associate stuff.
+
* Create a VM by importing the created qcow2 disk image, with <tt>virt-manager</tt>.
 +
* You may need to set manually the driver type to 'qcow2' in "disk" device section in the VM xml file (in /etc/libvirt/qemu/). In this case you need to redefine the VM xml file by: `sudo virsh -c qemu:///system define /etc/libvirt/qemu/ffsandbox.xml` then start the VM.
 +
* You need to reconfigure network interfaces in <tt>/etc/network/interfaces</tt> afterwards for replicating <tt>eth0</tt> config stanza for <tt>eth1</tt> for instance. You can find which device to enable by running <tt>ifconfig -a</tt>.
 +
* You may need to check the <tt>web_host</tt> setting in <tt>/etc/fusionforge/config.ini.d/debian-install.ini</tt> to adjust the forge's URLs for redirections.
  
* Avoiding Firefox self-signed certificate error dialogs when running selenium tests:
+
=== Running the test suite ===
** Start connection with valid DISPLAY (using the VM display, or ssh -X)
+
 
** Close your existing Firefox windows (if any) and run:
+
We need a few manual steps to avoid Firefox self-signed certificate error dialogs when running selenium tests.
 +
 
 +
* Start connection with valid DISPLAY (using the VM display, or ssh -X)
 +
* Close your existing Firefox windows (if any) and run:
 
  firefox -ProfileManager
 
  firefox -ProfileManager
** Configure a new Firefox profile in a directory of your choice, e.g. <tt>/root/fusionforge-trunk/tests/ff-profile-selenium/</tt>
+
* Configure a new Firefox profile in a directory of your choice, e.g. <tt>/root/fusionforge-trunk/tests/ff-profile-selenium/</tt>
** Still in Firefox, visit https://forge.local/ and add an exception for the self-signed certificate
+
* Still in Firefox, visit https://forge.local/ and add an exception for the self-signed certificate
** Close Firefox
+
* Close Firefox
** Go to <tt>/root/fusionforge-trunk/3rd-party/selenium/</tt>
+
* Go to <tt>/root/fusionforge-trunk/3rd-party/selenium/</tt>
** To run the tests with the corresponding Firefox profile, change the <tt>run</tt> rule of <tt>Makefile</tt> to:
+
* To run the tests with the corresponding Firefox profile, change the <tt>run</tt> rule of <tt>Makefile</tt> to:
 
  LANG=C java -jar selenium-server.jar -firefoxProfileTemplate /root/fusionforge-trunk/tests/ff-profile-selenium/ -interactive
 
  LANG=C java -jar selenium-server.jar -firefoxProfileTemplate /root/fusionforge-trunk/tests/ff-profile-selenium/ -interactive
** Run <tt>/root/scripts/start-selenium.sh</tt> in a terminal (will run the Makefile you just modified)
+
* Run <tt>/root/scripts/start-selenium.sh</tt> in a terminal (will run the Makefile you just modified)
** <tt>/root/scripts/run-testsuite.sh</tt> in another terminal (may take a few minutes to start)
+
* <tt>/root/scripts/run-testsuite.sh</tt> in another terminal (may take a few minutes to start)
  
* You may prefer running the X display inside the VM to start the selenium server (run with ./start-selenium.sh), so that windows opened by the tests don't bother you, rather than through a ssh -X made to the VM.
+
You may prefer running the X display inside the VM to start the selenium server (run with ./start-selenium.sh), so that windows opened by the tests don't bother you, rather than through a ssh -X made to the VM.
  
* Running the latest version of the scripts to test the 5.1 branch may be as easy as :
+
=== Updating the VM scripts ===
 +
 
 +
Running the latest version of the scripts to test the 5.1 branch may be as easy as :
 
  wget -q -O - "https://fusionforge.org/scm/viewvc.php/*checkout*/branches/Branch_5_1/tools/VM-scripts/configure-scripts.sh?root=fusionforge" | sh -s Branch_5_1
 
  wget -q -O - "https://fusionforge.org/scm/viewvc.php/*checkout*/branches/Branch_5_1/tools/VM-scripts/configure-scripts.sh?root=fusionforge" | sh -s Branch_5_1
  # Note: the script is now at https://fusionforge.org/plugins/scmgit/cgi-bin/gitweb.cgi?p=fusionforge/fusionforge.git;a=blob_plain;f=tools/VM-scripts/configure-scripts.sh;hb=HEAD and needs to be updated.
+
  # Note: the script is now at https://fusionforge.org/plugins/scmgit/cgi-bin/gitweb.cgi?p=fusionforge/fusionforge.git;a=blob_plain;f=tools/VM-scripts/configure-scripts.sh;hb=HEAD
 +
# and needs to be updated to use the new bzr-buildpackage repo.
 +
 
 +
=== Saving disk space ===
  
* If you want to install the GUI and lack disk space, it may help to:
+
If you want to install the GUI and lack disk space, it may help to:
 
  echo 'APT::Install-Recommends "false";' > /etc/apt/apt.conf.d/00no_recommends
 
  echo 'APT::Install-Recommends "false";' > /etc/apt/apt.conf.d/00no_recommends
 
  aptitude install xorg
 
  aptitude install xorg
 
  /root/scripts/install-gui.sh
 
  /root/scripts/install-gui.sh
  
* As of 2013-10, to install on Wheezy you'll want to manually install http://ftp.fr.debian.org/debian/pool/main/l/loggerhead/loggerhead_1.19~bzr479-3_all.deb and also <tt>moreutils</tt>, to fix dependencies.
+
You can study <tt>shrink.sh</tt> as well (to be run in <tt>telinit 1</tt>).
 +
 
 +
=== Wheezy deps ===
 +
 
 +
As of 2013-10, to install on Wheezy you'll want to fix a couple dependencies before running <tt>./install.sh</tt>:
 +
aptitude install moreutils
 +
wget http://ftp.fr.debian.org/debian/pool/main/l/loggerhead/loggerhead_1.19~bzr479-3_all.deb

Revision as of 11:55, 19 October 2013

Roland Mas has designed a virtual machine image (aka FusionForge Sandbox) to provide a development / test machine providing an already configured automated tests environment. FusionForge is not pre-installed but can be built using included scripts.

Download: http://fusionforge.fusionforge.org/sandbox/

See : http://lists.fusionforge.org/pipermail/fusionforge-general/2010-November/001245.html for more details

Presentation

Here's an excerpt from the enclosed /root/Desktop/README.html:


FusionForge sandbox

Welcome to the FusionForge sandbox. This VM/appliance is designed to help you get started on development. It's basically a clean Debian system (user:pass is root:root), with a checkout of the FusionForge code and a few helper scripts. Those scripts are in the /root/scripts directory:

  • update.sh will update the sources from FusionForge's Subversion repository, and also the currently installed packages.
  • build.sh builds FusionForge packages from the sources, and stores them in the package repository at /root/debian-repository. (rem Cbac starting from a debboostraped vm, needs to add dupload, bzr-svn and svn).
  • install.sh installs those FusionForge packages. This should end up with an up-to-date FusionForge running on http://forge.local/.
  • start-selenium.sh gets and runs Selenium, which is needed for the testsuite.
  • run-testsuite.sh actually starts the testsuite.
  • install-gui.sh sets up an X11 graphical environment.

The actual source code is stored under /root/fusionforge-trunk, so that's where you should do the changes you want to test.

Running everything on the VM

If you run install-gui.sh, then you get a graphical desktop environment on the VM. To build and install from sources, run the “Build + inst” script (there's an icon on the desktop). You can then use http://forge.local/, with the administrator's login being “admin” and its password being “myadmin”. To run the Test Suite, launch Selenium and keep it running (possibly iconized) while you launch the “Run testsuite”.

Running remotely

If you don't want to install a full GUI on the VM, you can also run the interesting things remotely through SSH. Depending on the configuration of your VM, the network settings may change, but the VM displays its IP address at boot time, so you can ssh root@192.168.x.x and work in there. Note that Selenium needs an X11 display (since it drives a graphical web browser), so you'll need to run ssh -X root@192.168.x.x start-selenium.sh (note the -X).


Note: the 2012-07 image uses an absolete bzr-svn repository, replace it with:

cd /root/
mv fusionforge-trunk fusionforge-trunk.old
bzr checkout https://fusionforge.org/anonscm/bzr/deb-packaging/unstable fusionforge-trunk

You may use this pre-configured VM image with Virtualbox (or convert it to another VM system), or install a development environment on an existing Debian Wheezy system using the VM-scripts/configure-scripts.sh script (see below).

Tips

To reset the admin password

Use: /usr/share/gforge/bin/forge_set_password admin

Converting from Virtualbox image format to qemu QCOW2

Converting from Virtualbox image format to qemu QCOW2: convert to raw format first

VBoxManage clonehd -format RAW FFSandbox-disk1.vmdk ffsandbox.raw  # ~4GB

Then convert to QCOW2 format:

qemu-img convert -f raw -O qcow2 ffsandbox.raw ffsandbox.qcow2

Running inside qemu

  • Create a VM by importing the created qcow2 disk image, with virt-manager.
  • You may need to set manually the driver type to 'qcow2' in "disk" device section in the VM xml file (in /etc/libvirt/qemu/). In this case you need to redefine the VM xml file by: `sudo virsh -c qemu:///system define /etc/libvirt/qemu/ffsandbox.xml` then start the VM.
  • You need to reconfigure network interfaces in /etc/network/interfaces afterwards for replicating eth0 config stanza for eth1 for instance. You can find which device to enable by running ifconfig -a.
  • You may need to check the web_host setting in /etc/fusionforge/config.ini.d/debian-install.ini to adjust the forge's URLs for redirections.

Running the test suite

We need a few manual steps to avoid Firefox self-signed certificate error dialogs when running selenium tests.

  • Start connection with valid DISPLAY (using the VM display, or ssh -X)
  • Close your existing Firefox windows (if any) and run:
firefox -ProfileManager
  • Configure a new Firefox profile in a directory of your choice, e.g. /root/fusionforge-trunk/tests/ff-profile-selenium/
  • Still in Firefox, visit https://forge.local/ and add an exception for the self-signed certificate
  • Close Firefox
  • Go to /root/fusionforge-trunk/3rd-party/selenium/
  • To run the tests with the corresponding Firefox profile, change the run rule of Makefile to:
LANG=C java -jar selenium-server.jar -firefoxProfileTemplate /root/fusionforge-trunk/tests/ff-profile-selenium/ -interactive
  • Run /root/scripts/start-selenium.sh in a terminal (will run the Makefile you just modified)
  • /root/scripts/run-testsuite.sh in another terminal (may take a few minutes to start)

You may prefer running the X display inside the VM to start the selenium server (run with ./start-selenium.sh), so that windows opened by the tests don't bother you, rather than through a ssh -X made to the VM.

Updating the VM scripts

Running the latest version of the scripts to test the 5.1 branch may be as easy as :

wget -q -O - "https://fusionforge.org/scm/viewvc.php/*checkout*/branches/Branch_5_1/tools/VM-scripts/configure-scripts.sh?root=fusionforge" | sh -s Branch_5_1
# Note: the script is now at https://fusionforge.org/plugins/scmgit/cgi-bin/gitweb.cgi?p=fusionforge/fusionforge.git;a=blob_plain;f=tools/VM-scripts/configure-scripts.sh;hb=HEAD
# and needs to be updated to use the new bzr-buildpackage repo.

Saving disk space

If you want to install the GUI and lack disk space, it may help to:

echo 'APT::Install-Recommends "false";' > /etc/apt/apt.conf.d/00no_recommends
aptitude install xorg
/root/scripts/install-gui.sh

You can study shrink.sh as well (to be run in telinit 1).

Wheezy deps

As of 2013-10, to install on Wheezy you'll want to fix a couple dependencies before running ./install.sh:

aptitude install moreutils
wget http://ftp.fr.debian.org/debian/pool/main/l/loggerhead/loggerhead_1.19~bzr479-3_all.deb