Select the search type
  • Site
  • Web
Search
You are here:  Support/Forums
Support

Bring2mind Forums

HELP - UI broken by large number of files in folder!!!
Last Post 02/22/2011 9:56 PM by Peter Donker. 5 Replies.
Sort:
PrevPrev NextNext
You are not authorized to post a reply.
Author Messages
Graham Lewis
New Member
New Member
Posts:21


--
02/17/2011 8:09 AM
Please help - one of my customers has a DMX installation in which they did a bulk import of over 5000 files into a single folder. Now we cannot access that folder to manipulate it. Whenever we try to access it we get the 'A script on this page is causing IE to run slowly' error. Even if I put the fix from http://support.microsoft.com/kb/175500 in place although it will show us the files we cannot do anything with them - if you click on any one of them the system just never responds. In Firefox the folder is completely inaccessible.

Lots of questions arising from this but these are the immediate ones:

1. I know the IE script warning is based on the number of statements. The limit is 5 million by default. The number of statements is presumably directly proportional to the number of files in the folder. So can you advise how many files is going to break the script statement limit?

2. I am trying to fix the problem by creating a bunch of new (sub)folders and moving the files there in small batches. But I can't do this through the GUI. Is there a simple way in the SQL tables to hack the folder for a file? If so please advise tables and columns I need to be looking at to action this.

3. Can we get an enhancement to the import that will prevent (or at least warn) if the import would result in this situation reoccurring?

TIA,
Graham
Graham Lewis
New Member
New Member
Posts:21


--
02/17/2011 9:53 AM
I have been doing some digging - here are my conclusions so far:

Everything I want to work with lives in the DMX_Entries table - there are no separate tables for folders and files.

The EntryID is the primary key.

A folder has an EntryType of Collection - the only easy way I can see to identify which one is which is to use keywords which live (logically enough) in the Keywords column.

Files have an EntryType starting 'File.' and the name displayed is held in the OriginalFileName column.

The folder structure is maintained by the Path column in every entry. It is constructed as a semi-colon separated list of EntryIDs starting with 0; (which I assume means root) and terminating with its own entry ID. So far example file EntryID 2003 sitting in a folder EntryID 32, that is in turn a sub-folder of EntryID 2 would have a path of '0;2;32;2003'.

So it seems to me that if I want to create a series of sub-folders below this folder 32 and move batches of files into them, I will need to do something like:

Identify each folder and record EntryID.

For each file I want to move insert into the Path value the EntryID for the new folder, so for the above file, if the new folder's EntryID is 8660 I edit the Path from '0;2;32;2003' to read '0;2;32;8660;2003'.

Which is fine as far as it goes but I have over 5000 files in this folder so I need to not have to do each one manually.

So can I just edit the Path for the files I want to move (I can fortunately identify them using their OriginalName) from '0;2;32;nnnn' to '0;2;8660;nnnn' by using REPLACE ';2;' by ';2;8660;'? Is the data type going to allow this or do I need a different SQL construct?

Peter can you please confirm that my conclusions are correct here. I am of course going to take the site offline and backup the database before trying anything like this, but some reassurance (or ideas to fix the original issue) would be much appreciated.

Regards,
Graham

Graham Lewis
New Member
New Member
Posts:21


--
02/17/2011 9:56 AM
Sorry, the REPLACE in the above should have read REPLACE ';32;' BY '32;8660;' of course...
Peter Donker
Veteran Member
Veteran Member
Posts:4536


--
02/18/2011 3:04 PM
Hi Graham,

Set the CollectionId field, not the Path field. The latter is calculated. The first determines where the entry goes. After running your SQL check that the Path values have updated. If not then run the srcipt to "repair structure" which basically does a sanity check on the Path field and makes sure it is OK again.

Peter
Graham Lewis
New Member
New Member
Posts:21


--
02/21/2011 11:51 AM
Peter,

Thanks for that. It's amazing how you can miss the obvious when working late at night under stress! As it turns out I managed to write SQL to update both columns reasonably effectively, I only finished up with about 50 files I had to edit manually which wasn't too bad. Site is back on the air now.

Do you (or any other forum readers) have any feedback to offer on my original query as to how many files in a folder is too many and trips the IE script limit? From my experience I can record that around 1000 is fine and 5000 is definitely bad, but that's still a wide range. Also, what about a warning on the import - I think it might be a useful, albeit not urgent, enhancement?

Regards,
Graham
Peter Donker
Veteran Member
Veteran Member
Posts:4536


--
02/22/2011 9:56 PM
Hi Graham,

This is not an exact science. Ajax relies on callbacks to the server. If they take too long the browser assumes the JS that encapsulates the callback is "taking too long" and begins to nag you. This means the messages appear as a function of amount of data, server/sql speed, line speed/bandwidth, client speed. There's not a single one that you can point a finger to and say: o, keep that below level XYZ.

Peter
You are not authorized to post a reply.