Filestore2 files recovered

Finally after being lost in limbo since upgrading to drupal 5, I've managed to recover the old files uploaded to the website.

Drupal is a great tool, but I've learned the hard way that you really have to be careful what contributed modules you rely on. 4 years ago when I first built the site in drupal 4.5, I decided to use a module that was not long for this world. after upgrading from 4.5 to 4.6 (where filestore became filestore2) and then to 4.7, the filestore2 and fscache modules ceased development. This left us with a ton of data that was not accessible (ok not a ton, but 77 nodes/files from the early days of the project).

At last weekend's NYC Drupal Camp #5, Joe Golden and I started working on a perl script that would recover the files and move the nodes from filestore2 to standard drupal page nodes with file attachments.

What follows is the end result, it's also attached to this post if you want to download it and use it.

Thanks Joe for the motivation to finally get this task out of the way!

# This is for converting filestore2 nodes/files into attachments to page nodes
# Use at your own risk
# written to recover files for the website
# by Joe Golden and Eric Goldhagen
# code licensed under:
# Version 2, June 1991

# connect to database and format results
use DBI;
my $dsn = "dbi:mysql:databasename:localhost:3306";
my $dbh = DBI->connect($dsn, "username", "password");
#initialize various variables
my $number_changed="0";
my $starting_revision="0";
my $starting_revisions="";
my $howmanyfiles="0";

#directories relative to location of this script
#set the path to fscache directories
my $oldpath="misc/fscache/fscache/";
#set path to dir drupal uploads files to now
my $newpath="documents/";

#get starting number for fid
my $starting_fids="";
my $starting_fid="";
$sth=$dbh->prepare("select id from sequences where name='files_fid'");

#select information to be used in moving files
print $starting_revision."--start at\n\n";
$sth2= $dbh->prepare("select filestore2.nid, fscache.filename, fscache.cachename, fscache.mimetype, fscache.size from filestore2 join fscache on filestore2.fsid = fscache.fsid");
#loop through results
while (@data = $sth2->fetchrow_array()) {

my $filename, $path_stored, $realpath;

my $nid=$data[0];
my $file=$data[1];
my $cachename=$data[2];
my $filemime=$data[3];
my $filesize=$data[4];

$filename = $file;
$path_stored = $cachename;
if ( $path_stored =~ /(..)(..)(....)/ ){
$realpath = $oldpath.$1."/".$2."/".$path_stored;
# print "real path is $realpath\n";
} else {
print "badd pattern". $nid."\n";
my $cleanfilename = $filename;
$cleanfilename =~ s/ /\\ /go;
my $filemover= "cp ".$realpath." ".$newpath.$cleanfilename;
my $newfilepath=$newpath.$filename;
print "file: ".$realpath." moved to ".$newpath.$filename."\n";

#increment the counter for files

#db query to insert a new file attachment
#for the right node
my $constructed_statement ='insert into files (fid,nid,filename,filepath,filemime,filesize) values ('.$starting_fid.','.$nid.',"'.$filename.'","'.$newfilepath.'","'.$filemime.'","'.$filesize.'")';
$sth = $dbh->prepare($constructed_statement);

#db query for update of file_revision

#need vid of node for vid here
my $nodevids="";
my $nodevid="";

my $nodevidstatement="select vid from node where nid=".$nid;

my $constructed_statement='insert into file_revisions (fid,vid,description,list) values ('.$starting_fid.','.$nodevid.',"",1)';
print $constructed_statement."\n";

print "attached to node:". $nid." revision:".$nodevid."\n\n";

#increment the fid

#db query change info for node type after we're done moving everything
$sth=$dbh->prepare('update node set type="page" where type="filestore2"');

#insert our new node fid into sequence table

my $constructed_statement='UPDATE sequences set id='.$starting_fid.' where name="files_fid"';

print "-----------------------------------\n";
print "final fid is ".$starting_fid."\n";
print $howmanyfiles." files converted\n\n";
print "please check the output above for any errors that might have happened\n\n";


Thank you!

Hi Eric - I can't believe you just posted this. I am in the process of upgrading a client from Drupal 4.6 -> 6 and just realized that we have a big issue with converting from filestore2. Your script seems to be the solution. I'll give it a try and let you know how it goes. Thank you!

let me know how it goes!

good luck, let me know if anything is confusing or needs better documentation. Note that the script is written explicitly against the mysql database schema for drupal 5, so run it when you are at that point of the upgrade.



Just to let you know, I finally made the time to get back to this upgrade and your script worked perfectly. Thanks for pointing out that it needed to run against Drupal 5, as I was originally planning on running it at 4.7.

Thanks again!

Filestore2 files recovered


Your post has helped me to recover my old files. Thanks for the script.