Welcome to the new BlahBlahCafe!

A new server, a new environment. Updated and all.

As with every migration, some things might have been lost in translation (at least, that's my experience in software development). The main problem is logging in into the boards. These could be caused by several reasons:

- Your browser cache & cookies need to be cleared because we moved to a new server. If you get the error message whilst logging: "The submitted form was invalid. Try submitting again.", you need to clear your cache & cookies. You can find a "Delete cookies" option at the bottom of this page.

- There is also a chance that your password is not correctly converted to the new system. This is a known bug with migrating the system. The only solution is to request a new password, there's a link on the form that will send an email to you to create a new password. If that fails, I can set a password for you, you can contact us at the "Contact us" link at the bottom of the page.

The current state of the forum

Got ideas? Suggestions? Problems? Error messages? Share 'em here!

Moderator: L'equipe Zoolook

User avatar
Dr_Jones
Posts: 2035
Joined: Thu Mar 30, 2006 1:06 pm

The current state of the forum

Post by Dr_Jones »

The story so far.

There's currently three problems with the forum.
1. It is running on a PHP5.x server with a altered version of the phpbb2-forum software called Przemo
2. Somewhere on august 25, there was a crash that caused the forum to stop working. As a result, two tables are completely gone from the forum database. I'll explain this later.
3. The posts-counter has been reset to 0. I will also explain this later, because it has a correlation with 2.

Ok, how to solve these problems?

1. Running the Przemo software on php7 is not possible. Upgrading from phpbb2 -> phpbb3 seems like the only possible solution, phpbb3 is running on php7.
phpbb3 offers an option to import your phpbb2 forum data to phpbb3. I've tested this and as a first result, it seems to be working okay. Of course, there's the usual UTF-8/unicode problems (diacretes, emoji's, etc), but I think that is something I can overcome. Logging in works with my zoolook.nl account, so user data works, and I can see my private messages too. Jay!

2. The missing data. This is the real problem. All topic data is gone and all post data is gone. However, and that's a bit of good news, all the post text still exists.

What does that mean?
Well, if you look at the board hierarchical, you have the following structure

Forums -> topics -> posts -> posts text

This is how it is modeled in the database too.

The forums are News, Oxygene, Articles, etc. Within these forums there are the topics, with in the topics there are the posts.

You can see the forums just fine now, but all the topics are gone. The topic table contains (amongst other things) the topic title and a link to which forum it belongs to.
The posts table is similar, it has a reference to the topic it belongs to and a reference to the forum it belongs to. A bit of redundancy there. It has also a user-id of the one who posted, a datetime value, and many other things. But the posts table does not contain the posts text, that is stored in a different table.
This means there are a bit more than 200,000 orphaned posts. And judging from the highest forum-ID, it is also missing 7,000 topics.

I've tried to get a backup database from the webhosting and it turns out they have a backup of maximum of 7 days old. That means we're a few days too late. This hurts, of course.

Beatle is going to look if there's a backup somewhere on a local disk, but we don't know if there's one and if there is, how old it is.

I've tried to figure out if data could be restored by using some redundancy values, but there's too much data lost. If the topics table only was gone, you could restore it quite easily.

3. I've reset the auto-increment number to the latest post + 1. That should work, I hope. We'll see when I post this message. :-)

Okay, now for what to do next.

I will focus on restoring the topics and posts. If there's no backup, we will have a problem, but I think all is not lost. Firstly, we always have the posts text, that is not gone. I'm hoping people have this forum mirrored somewhere. So if there's someone who has downloaded everything from this forum, let me know. I might restore data from the HTML fiiles.

In the meantime, I can try to port the data to phpbb3 without the utf-8 problems. My deadline for a 100% correct port is 15 september.

Any suggestions and ideas are welcome.
User avatar
Adam
Posts: 193
Joined: Thu Mar 30, 2006 8:24 pm
Location: Cornwall,U.K.
Contact:

Post by Adam »

I sucessfully converted a phpbb2 forum to phpbb3 some years ago using their built in converter.
If there is any way you could send me a copy of the forum's SQL database there are a couple of things I'd like to try.
My own forum runs on phpbb 3.2.7 and it would interesting to compare to my database to see how it inserts posts and topics, I can try and convert it but can't guarantee that it will work, but at the end of the day anything is worth a try
User avatar
Dr_Jones
Posts: 2035
Joined: Thu Mar 30, 2006 1:06 pm

Post by Dr_Jones »

see your PM.
User avatar
Dr_Jones
Posts: 2035
Joined: Thu Mar 30, 2006 1:06 pm

Post by Dr_Jones »

I've done a successful import of an old version of the forum, at least, that's what I think. I counted the users from before the import and after the import and excluding the bots all users were imported, I think. The amount of private messages seems to be the same too, so I guess not much data is gone from the import.

Now for the missing forum and posts tables.. this will be a problem. From the Google cache, I've seen we got 6844 topics in July and a total of 202817 posts. As of now, all these 202817 (and more, because August is not included) are orphaned. We're trying everything to get the data and the connection between forums, topics and posts back. This includes using Google cache, archive.org, the access-logs and other means.

Speaking of which: does anyone of you have a mirror of the forum by any chance? Someone who's wget-ed the forum. This might be most useful.
User avatar
Dr_Jones
Posts: 2035
Joined: Thu Mar 30, 2006 1:06 pm

Post by Dr_Jones »

Today's update:
Not much today, it's a regular working day for me.

But I've got some help from Adam and 1906, very much appreciated! So I retrieved all topics from the beginning up to december 2006 from an old backup, that means we've only got 14 years to catch up.

Adam is working on the current database and is trying to get a phpbb2 version up and running for testing purposes. I've gotten some useful tips from him and he did some testing on my version of the board. His suggestion to use some extensions resulted in a working shoutbox on the new version, yay! Adam also made a call on facebook if there are people out there who have a mirror of the website somewhere.

1906 and me were trying to find some ways to get the old database-data back, but to no avail. We've thought about some clever ways to reverse engineer things from the Google cache, let's hope this will work. We've thought of some other plans which I've set in motion, but those will be long and time-consuming tasks, I'm afraid.
User avatar
Dr_Jones
Posts: 2035
Joined: Thu Mar 30, 2006 1:06 pm

Post by Dr_Jones »

Update: I did not have much time today again (working day again), but I did a quick analysis on the archive.org data. I could restore some topics so that there's now 2988 topics. There used to be 6480 topics, so that's 46%.

The hardest part will be connecting the posts with the topics. I could manage a handful of them so 39885 posts are connected with topics. That's 19%.

Again: mirrors of the websites would be fantastic, these could help a lot with restoring lost topics and the connections between topics and posts.

I'm going to take a few days off now, I also have a social life :-)
User avatar
Kanta
English Moderator & Miss News
Posts: 8616
Joined: Thu May 04, 2006 12:08 pm
Location: United Kingdom

Post by Kanta »

Thank you for all your hard work. Much appreciated.
Jarregirl YouTube
Concerts attended:
Théâtre Marigny, Paris - 2007
Symphony Hall, Birmingham - 2008
RAH, London - 2008
Wembley Arena, London - 2009
NIA, Birmingham - 2009
POP Bercy, Paris - 2010
NIA, Birmingham - 2010
O2 Arena, London - 2010
Zénith Aréna, Lille - 2010
Port Hercule, Monaco - 2011
TUI Arena, Hannover - 2011
Festival International de Carthage - 2013
Barclaycard Arena, Birmingham - 2016
User avatar
Dr_Jones
Posts: 2035
Joined: Thu Mar 30, 2006 1:06 pm

Post by Dr_Jones »

Thanks people (also from the shoutbox), it's a lot of work, but it is fun. Digital archeology (hence the name Dr_Jones :-D) And Kanta, digging through all the old posts, I really admire all the work you have done already!
User avatar
Velodynamic
Posts: 3004
Joined: Thu Mar 30, 2006 7:32 pm
Location: Sweden, far north
Contact:

Post by Velodynamic »

Even if you don't find topics and posts from the forum's first ten years or so it's still more then we can ask for I think. You're doing a fantasic job Doc.
Maybe it works more randomly then in chronological order? I dunno...
This board has been like a cat with eleven lives in the past. :smoke:
"The worst thing in any form of art is to be betrayed by your own habits."
- JEAN MICHEL JARRE
User avatar
Kanta
English Moderator & Miss News
Posts: 8616
Joined: Thu May 04, 2006 12:08 pm
Location: United Kingdom

Post by Kanta »

Thanks people (also from the shoutbox), it's a lot of work, but it is fun. Digital archeology (hence the name Dr_Jones :-D) And Kanta, digging through all the old posts, I really admire all the work you have done already!
Thank you. Much appreciated for your very kind words. :D
Jarregirl YouTube
Concerts attended:
Théâtre Marigny, Paris - 2007
Symphony Hall, Birmingham - 2008
RAH, London - 2008
Wembley Arena, London - 2009
NIA, Birmingham - 2009
POP Bercy, Paris - 2010
NIA, Birmingham - 2010
O2 Arena, London - 2010
Zénith Aréna, Lille - 2010
Port Hercule, Monaco - 2011
TUI Arena, Hannover - 2011
Festival International de Carthage - 2013
Barclaycard Arena, Birmingham - 2016
User avatar
Adam
Posts: 193
Joined: Thu Mar 30, 2006 8:24 pm
Location: Cornwall,U.K.
Contact:

Post by Adam »

And I now have a working copy of the forum as it is now (Przemo's phpbb2) running on my PC for testing purposes via Wamp server, which allows you to work on any website offline & I have been uploading some of the restored posts to the database via scripts from Dr Jones.
I will also look into building a JMJ themed template for phpbb3 based on a template I have already built for my own forum, which will take a while but no hurry as the priority at the moment is restoring missing posts.
User avatar
Dr_Jones
Posts: 2035
Joined: Thu Mar 30, 2006 1:06 pm

Post by Dr_Jones »

Update:
Not much work on the phpbb3 version of the forum today. I will work on that the coming days.

For now, I've worked on digging through old posts via the Google cache and Bing cache. And what do you know? Microsoft beats Google in user-friendlyness! I can get posts automatically from Bing's cache, while Google has anti-bot measures, which are highly annoying.

Anyway, that way I have managed to get the following results:
Total posts 58512 Total topics 3714

That means I've so far restored 54% of the topics data! I did not expect this result at all! Post data is restored for 28%, which is okay.
Topics will have "holes" in them, the longest running threads are a lot less longer, sadly. These are usually the most interesting ones, like the new albums threads, tour threads, ebay thread, etc. I guess we have to live with that. Besides, there's a possibility that I can restore topics to the original topic via quotes. Remember that all the texts are still in the database, so matching might be possible that way. We're thinking of ways to do that.

Here's an screenshot of my development-system:
Image

It's using the standard subsilver template, as Adam said, customizing a template is a low priority right now, but I'm looking forward into seeing that happen.
User avatar
Kanta
English Moderator & Miss News
Posts: 8616
Joined: Thu May 04, 2006 12:08 pm
Location: United Kingdom

Post by Kanta »

Splendid achievement by both of you.
Jarregirl YouTube
Concerts attended:
Théâtre Marigny, Paris - 2007
Symphony Hall, Birmingham - 2008
RAH, London - 2008
Wembley Arena, London - 2009
NIA, Birmingham - 2009
POP Bercy, Paris - 2010
NIA, Birmingham - 2010
O2 Arena, London - 2010
Zénith Aréna, Lille - 2010
Port Hercule, Monaco - 2011
TUI Arena, Hannover - 2011
Festival International de Carthage - 2013
Barclaycard Arena, Birmingham - 2016
User avatar
Dr_Jones
Posts: 2035
Joined: Thu Mar 30, 2006 1:06 pm

Post by Dr_Jones »

Last night I've worked on the phpbb3 version. I did some testing on registering for a new user and that worked fine. I got an activation-email, so it looks all works good!
I also worked on transferring the avatars to the new system. The filenames have been changed in the new version, so I've renamed them all.

Today, I expect to get some matched records from 1906 from the printview pages. These contain a full topic, but lack topic and post-ids. That means you have to match via other means. I'm really curious to learn how much can be saved from that.
User avatar
Dr_Jones
Posts: 2035
Joined: Thu Mar 30, 2006 1:06 pm

Post by Dr_Jones »

OK, latest update:

Since the end of the month is approaching, we have to chose what to do now. I have a forum running on PHP7 and PHPBB3. A part of the data is restored and the forum is functional. In fact, it's doing pretty well. I'm going to ask some people to take a look at it (not looking at the layout), to test if they can log in and all that. If these tests are okay, we can move the domain to the new forum and it's up and running.

In the meantime, we're working hard on a new style for the forum and restoring more of the lost posts.
Post Reply