Forum: help

Monitor Forum | Start New Thread Start New Thread
RE: [Running] Create new forge from (probably) V3.3 gforge database dump [ Reply ]
By: Franck Villaume on 2018-04-24 18:00
[forum:1125]
That is a good news! Well done!
Now you can apply update from beta2 to RC1 :-)

RE: [Running] Create new forge from (probably) V3.3 gforge database dump [ Reply ]
By: Graham Miller on 2018-04-23 04:03
[forum:1123]
With quite a journey I got the forge up and running with the old DB and (maybe) all the DB updates applied.

Running version 6.1.0beta2-1.201803251148

There was some selinux issues, some misunderstanding on my part, and a fair bit of manually applying SQL updates, and modifications to install-ng script.

I won't post the saga here ... its too big a story with a few stuff-ups on the way.

Thanks for your help.

RE: Create new forge from (probably) V3.3 gforge database dump [ Reply ]
By: Franck Villaume on 2018-04-06 08:55
[forum:1122]
no impact. it's better that way.

RE: Create new forge from (probably) V3.3 gforge database dump [ Reply ]
By: Graham Miller on 2018-04-06 07:28
[forum:1121]
Just had a thought.

I have cron and systasksd still disabled. Will that affect my attempting to run these scripts? It does not look like it from where I sit, but I just thought I'd ask?

RE: Create new forge from (probably) V3.3 gforge database dump [ Reply ]
By: Graham Miller on 2018-04-05 02:38
[forum:1120]
When running 20090507-install_workflow.php I get an error

[graham@jenshen2 db.511]$ sudo -u fusionforge php 20090507-install_workflow.php
PHP Fatal error: Call to undefined function db_query_params() in /usr/share/fusionforge/db.511/20090507-install_workflow.php on line 18

Can't find any logs for this one either.

A bit stuck now. Wish I could find the logs to see why these cli scripts are failing.

RE: Create new forge from (probably) V3.3 gforge database dump [ Reply ]
By: Graham Miller on 2018-04-05 02:30
[forum:1119]
About startpoint.php, I cannot run it from the db.511 directory OR the v6.1 db directory. They both give me the following errors:-
[graham@jenshen2 db.511]$ php startpoint.php
PHP Notice: Undefined variable: gfconn in /usr/share/fusionforge/db.511/startpoint.php on line 7
PHP Fatal error: Call to undefined function db_error() in /usr/share/fusionforge/db.511/startpoint.php on line 8
[graham@jenshen2 db]$ sudo -u fusionforge ./startpoint.php
PHP Notice: Undefined variable: gfconn in /usr/share/fusionforge/db/startpoint.php on line 27
PHP Fatal error: Call to undefined function db_error() in /usr/share/fusionforge/db/startpoint.php on line 28

[graham@jenshen2 db]$ php -v
PHP 5.4.16 (cli) (built: Mar 7 2018 13:34:47)

I have enabled development mode and turned on most of the debug variables, but I can't find where the log is?

There does not seem to be anything going to /var/log/fusionforge/ or /var/log/httpd/ or /var/log/messages.

Here is the config list:
$ sudo -u fusionforge forge_get_config list-all-variables-values
core/forge_name: FusionForge
core/source_path: /usr/share/fusionforge
core/gettext_path: /usr/share/locale
core/binary_path: /usr/share/fusionforge/bin
core/plugins_path: /usr/share/fusionforge/plugins
core/data_path: /var/lib/fusionforge
core/config_path: /etc/fusionforge
core/log_path: /var/log/fusionforge
core/apache_user: apache
core/apache_group: apache
core/apache_service: httpd
core/ssh_service: sshd
core/extra_config_dirs: /etc/fusionforge/config.ini.d
core/installation_environment: development
core/sysdebug_enable: 1
core/sysdebug_phphandler: 1
core/sysdebug_backtraces:
core/sysdebug_ignored:
core/sysdebug_dberrors: 1
core/sysdebug_dbquery:
core/sysdebug_xmlstarlet:
core/url_root: /usr/share/fusionforge/www/
core/themes_root: /usr/share/fusionforge/www/themes
core/chroot: /var/lib/fusionforge/chroot
core/homedir_prefix: /var/lib/fusionforge/chroot/home/users
core/groupdir_prefix: /var/lib/fusionforge/chroot/home/groups
core/upload_dir: /var/lib/fusionforge/download
core/ftp_upload_dir: /var/lib/fusionforge/ftproot
core/scm_snapshots_path: /var/lib/fusionforge/scmsnapshots
core/scm_tarballs_path: /var/lib/fusionforge/scmtarballs
core/custom_path: /etc/fusionforge/custom
core/lists_host: lists.forge.lan.webs.net.au
core/scm_host: scm.forge.lan.webs.net.au
core/forum_return_domain: forge.lan.webs.net.au
core/users_host: users.forge.lan.webs.net.au
core/admin_email: webmaster@forge.lan.webs.net.au
core/account_manager_type: pgsql
core/compression_method: gzip
core/default_country_code: US
core/default_language: English
core/default_theme: funky
core/default_timezone: GMT
core/default_trove_cat: 18
core/force_login:
core/mailman_path: /usr/lib/mailman
core/mailman_data_path: /var/lib/mailman
core/mailman_cgi_path: /usr/share/fusionforge/lists/cgi-bin
core/news_group: 2
core/peer_rating_group: 4
core/project_registration_restricted: 1
core/project_auto_approval:
core/project_auto_approval_user: admin
core/sendmail_path: /usr/sbin/sendmail
core/show_source:
core/stats_group: 3
core/template_group: 5
core/unix_cipher: SHA512
core/url_prefix: /
core/use_activity: 1
core/use_docman: 1
core/use_forum: 1
core/use_forum_mail_replies:
core/use_frs: 1
core/fti_config: fusionforge
core/use_ftp: 1
core/use_gateways: 1
core/use_mail: 1
core/use_manual_uploads: 1
core/use_news: 1
core/use_people: 1
core/use_pm: 1
core/use_project_multimedia:
core/use_project_vhost: 1
core/use_ratings: 1
core/user_registration_restricted:
core/user_notification_on_activation:
core/use_scm: 1
core/use_scm_snapshots: 1
core/use_scm_tarballs: 1
core/allow_multiple_scm:
core/use_shell: 1
core/use_shell_limited:
core/use_snippet: 1
core/use_ssl: 1
core/use_survey: 1
core/use_tracker: 1
core/use_trove: 1
core/use_project_tags: 1
core/use_project_full_list: 1
core/sitestats_projects_count: visible
core/allow_project_without_template: 1
core/use_webdav:
core/user_default_shell: /bin/bash
core/user_display_contact_info: 1
core/restrict_users_visibility:
core/forge_homepage_widget:
core/use_quicknav_default: 1
core/use_home: 1
core/use_my: 1
core/check_password_strength:
core/use_object_associations:
core/use_tracker_widget_display:
core/use_docman_review:
core/session_expire: 3600
core/use_artefacts_dependencies:
core/use_user_theme: 1
core/scm_single_host: 1
core/system_user: fusionforge
core/system_user_ssh_akc: fusionforge_ssh_akc
core/apache_auth_realm: SCM for FusionForge
core/users_default_gid: 100
core/web_host: forge.lan.webs.net.au
core/bin_dir: /usr/share/fusionforge/bin
core/shell_host: forge.lan.webs.net.au
authbuiltin/plugin_status: valid
authbuiltin/required:
authbuiltin/sufficient: 1
authbuiltin/sync_data_on: never
globalactivity/plugin_status: valid

RE: Create new forge from (probably) V3.3 gforge database dump [ Reply ]
By: Graham Miller on 2018-04-05 00:54
[forum:1118]
About v3.3 - I have not looked for v3.3 code, but it is mentioned in the v5.1.1 startpoint.php in the check_version() method. I do have a downloaded file in our archives:
- gforge-3.3.tar.bz2
I am happy to upload it or send it to you if you want it for your archives.

I ran the SELECT statement you suggested and it returns 0 rows. So I will run that script and see how it goes.

As for the database_changes table, I don't have one of those yet because I have not run startpoint.php so I will try to get that to run. Just have to decide on what is the start point.

According to the check_version() method, v4.8.2 is dated 20090402 so, even though I have gone past that date manually, I might set that as a startpoint then run the upgrade-db.php script to get to v5.1.1.

RE: Create new forge from (probably) V3.3 gforge database dump [ Reply ]
By: Franck Villaume on 2018-04-04 17:03
[forum:1117]
Hi,

about 3.3: good to know. I'm always discovering things about FusionForge. Can you point me to the 3.3 source code?

about the 20090507-install_workflow.php script:

run the following SQL query on your database:
SELECT group_id, artifact_group_list.group_artifact_id, element_id, artifact_extra_field_elements.extra_field_id
FROM artifact_extra_field_list, artifact_extra_field_elements, artifact_group_list
WHERE artifact_extra_field_list.extra_field_id=artifact_extra_field_elements.extra_field_id
AND artifact_group_list.group_artifact_id = artifact_extra_field_list.group_artifact_id
AND field_type=7

If this query returns 0 row, then you can run this script safelly.
If this query returns some rows, I've looked at the PHP code in the 6.1 branch, from my point of view, it should work as well.

About startpoint.php & upgradedb.php :
Did you insert into database_changes all the scripts that you run successfully? You have to.
Extract from the content of my database_changes:
scmhook:20180215-plugin_scmhook_multirepo_support.sql
scmhook:20180217-create_scmhook_commitemail_git_hg.sql
20160324-svn-prepare-multirepo.php
20180301-scm-stats-plugin-id.sql
20180302-scmhg-multirepository-support.php

RE: Create new forge from (probably) V3.3 gforge database dump [ Reply ]
By: Graham Miller on 2018-04-04 13:48
[forum:1116]
Hi,

Well now I have imported the old v4.5 database. Had to fix db object ownership to all fusionforge. It looks like I nearly have the DB at v5 but need some advice before trying the upgrade script in v6.1 (the orig installed ver).

I grabbed a v5.1.1 download and pulled the db/ directory to use on the v4.5 db. Copied to /usr/share/fusionforge/db.511/, cd into it, then tried the startpoint.php file but got errors and realised the structure in /common is different. Decided not to try that script.

So I went through, methodically, all the sql scripts from after v3.3 (yes there was mention of that version) and compared the DB (pgAdmin) to the effects of the upgrade statements. Got right up to v4.5 before finding any glaring differences. Applied the sqls in order until the end of 2009, where I got to a php script that needed to instantiate fusionforge objects.

20090507-install_workflow.php

It looks like this should be run before tackling the v6.1 upgrade.php script. The earliest upgrade sql script in v6.1 is 20100308-drop-forum-attachment-type.sql which, in the v5.1 upgrade scripts is the next task after the install_workflow.php.

Here are the require lines at the top of the install_workflow file:
require_once dirname(__FILE__).'/../www/env.inc.php';
require_once $gfcommon.'include/pre.php';
require_once $gfcommon.'tracker/Artifact.class.php';
require_once $gfcommon.'tracker/ArtifactFile.class.php';
require_once $gfwww.'tracker/include/ArtifactFileHtml.class.php';
require_once $gfcommon.'common/tracker/ArtifactType.class.php';
require_once $gfcommon.'tracker/ArtifactTypeFactory.class.php';
require_once $gfcommon.'tracker/include/ArtifactTypeHtml.class.php';
require_once $gfwww.'tracker/include/ArtifactHtml.class.php';
require_once $gfcommon.'tracker/ArtifactCanned.class.php';
require_once $gfcommon.'tracker/ArtifactExtraField.class.php';
require_once $gfcommon.'tracker/ArtifactExtraFieldElement.class.php';
require_once $gfcommon.'tracker/ArtifactWorkflow.class.php';

Do you think that v6.1 will instantiate those class well enough to allow the workflow to the installed? I just want to check before running this on the database I have so carefully groomed.

Yes I have a backup but would prefer to reduce the manhours going into this by checking first <grin>.

Hope you know the answer to this.

Thanks, Graham




RE: Create new forge from (probably) V3.3 gforge database dump [ Reply ]
By: Franck Villaume on 2018-03-01 15:12
[forum:1099]
Hi,

that is correct:
here is the link to upgrade scripts to 5.0.3:

https://scm.fusionforge.org/anonscm/gitweb?p=fusionforge/fusionforge.git;a=tree;f=gforge/db;h=8b43d4c34780047933d777fad48bdfe256528e87;hb=eeb1fe37667449ece62f03d59aec3d294829c546

And you are going to run into trouble :-)
Please post your feedback/errors here. I will try to help you to debug.


RE: Create new forge from (probably) V3.3 gforge database dump [ Reply ]
By: Graham Miller on 2018-03-01 10:45
[forum:1098]
Thanks for the good info Franck.

I do not have the source code of this old version so am only guessing about the version number. Shame its not stored in the DB somewhere.

I found various versions of gforge in our "download repo" archive, and guessed the version based on the file timestamp. Someone must have miss-named the file I guess.

So you are probably right about the 4.5.3, as there does not seem to be any downloads after that one.

I understand your instructions OK, so what I need to find is a version 5x of gforge/fusionforge to get the upgrage scripts from there and apply them to the revived archive version of the DB, correct?

Cheers, Graham

RE: Create new forge from (probably) V3.3 gforge database dump [ Reply ]
By: Franck Villaume on 2018-03-01 08:00
[forum:1097]
Hi Graham,

how I would do it:
- install a brand new fusionforge, including the database.
- stop all fusionforge services (apache + systasksd + cron)
- drop the brand new fusionforge database
- create an empty fusionforge database
- inject your database dump
- looks for migration script to do an incremental migration: See https://scm.fusionforge.org/anonscm/gitweb?p=fusionforge/fusionforge.git;a=heads to get migration scripts for the correct version.

The incremental migration objective is to transform progressively your data into the correct format.

About the gforge version you have:
v3.3 does not exist as far as I know.
See https://scm.fusionforge.org/anonscm/gitweb?p=fusionforge/fusionforge.git;a=tags
Ten years ago: I bet on 4.5.3 or a little bit above. Looking at the source code could help you to identify the version.

FusionForge 6.1 migration script works correctly if your database is 4.8.50 compatible. Therefore you have to upgrade your database from 4.5 to 5.0.

Looks at this page as well:
https://fusionforge.org/plugins/mediawiki/wiki/fusionforge/index.php/Migrating_very_old_Gforge

Hope this help.
Franck aka nerville

Create new forge from (probably) V3.3 gforge database dump [ Reply ]
By: Graham Miller on 2018-03-01 00:30
[forum:1096]
I have the PostgreSQL dump of the database for the gforge installation that managed a software development job and was moth-balled 10 years ago. I would like to get this project data into a new forge based on fusionforge. Will this be possible?

I do not have the source code from the live server at the time, but from the software packages downloaded around that time it is either gforge 3.3 or 4.5.3. My best guess would be gforge 3.3

There is custom data (what looks like project related data) in the following tables:-
doc_data
doc_groups
forum
forum_agg_msg_count
forum_group_list
forum_monitored_forums
forum_saved_place
frs_package
group_history
groups
news_bytes
project_group_list
project_history
project_task
trove_cat
trove_group_link
user_group
user_preferences
users
artifact_group_list
artifact_status
artifact
artifact_history
artifact_file
artifact_message
artifact_canned_responses
artifact_counts_agg
activity_log
project_category
project_messages
-- the above seem to be in the latest stable fusionforge
artifact_perm
artifact_category
artifact_group
-- these seem to be orphans

(I left out the stats* tables that had data but I don't care about them)

So I am guessing:-
- install a new fusionforge installation
- adapt the dump "COPY ... FROM stdin" sql scripts to suit fusion DB tables
- remove create table seq etc from DB dump file
- remove unwanted data from DB dump file
- run the import script.

So will this work? Am I on the right track?

Thanks, Graham