When we add new folders and/or files to the DMX document admin area they upload successfully but do not always get synchronized into the DNN file manager. Specifically, when browsing the DNN file manager we don't see the files there (sometimes for a while, sometimes EVER) and in turn we cannot use the files in places where we'd link to them from the DNN RadEditor while editing HTML content.

Patterns we noticed:

1. Some folders are pickier than others. We've renamed folders which seems to trigger actions that lead to the folder ultimately showing up in the DNN file manager later. (i.e. Inserting a sync record, triggering a sync immediately, fixing a broken or missing database row, or something along those lines.)

2. In some cases, renaming files may have done something similar to renaming folders (above, item #1)

3. Manual, recursive synchronization from the DNN file manager has helped. Sometimes though it takes 5+ attempts, it seems timing is really finicky and/or it doesn't work consistently.

4. According to DNN, if their File Manager is set to synchronize automaticaly, 3rd party document managers should work during this automated sync. Is this really the case? As we have this set under:

Host > Host Settings > Other Settings > File System Auto-Sync

To workaround we've used something that we really dislike. It makes using DMX only borderline acceptable. We directly grab a link from DMX via 'Direct Link':

1. upload new document into DMX folder
2. highlight document
3. right click and select 'show direct link'
4. copy the URL
5. go to the page where you want to create a link to the document
6. highlight text desired to link
7. go into hyperlink manager (on editor toolbar)
8. paste captured URL into the URL field
9. select 'new window'

I'd be willing to add a new entry to the DNN Scheduler, or patch DMX, etc. as long as we addressed the constant state of out-of-sync between DMX and DNN's file manager.

Being that we MUST use linking in the editor, which uses links from the DNN file manager tables, we have no solution at this point.

Any ideas?

The way this works is that DNN is the *active* party in this conversation. DMX has a class that inherits from DNN's FolderProvider class and this is announced in the web.config. Then, DNN checks all its folders once in a while (scheduled task). In doing so it may stumble on DMX folders. In this case it goes to the DMX class and calls a number of methods, such as GetSubFolders.

Now, the one snag is that DNN effectively copies this to its own tables and serves it out to anybody. I.e. there is no way for DMX to tell it that folder/file XYZ is only visible to role A or person B. So to avoid a gaping security hole, DMX only serves out public content (i.e. folders/files where "All Users" have view permission). Note that if a folder is not visible, files underneath it won't either.

I can't say from your description, above, where things go awry. Is there no error message in the event log of DNN? It is not something I've heard of or seen before, in any case.

According to DNN, if their File Manager is set to synchronize automaticaly, 3rd party document managers should work during this automated sync. Is this really the case?

Yes. But any scheduled task can be cut off by IIS. I don't expect that to happen, though. You could verify a successful completion in the scheduler page of DNN by checking the history.

Just out of curiosity: are we talking about a lot of data, here?

Peter

Hi Peter,
we have the same problem, but can't seem to get any luck with Files syncing, folders seem ok only sometimes, but no files showing in the File Manager, even though they are there in DMX.

We recently purchased the module for use on an Intranet, and it's just no good relying on the DNN File/Folder syncing. we used the module a few years ago and were using the DnnWerk RadEditor Provider and the DMX Browser. this is what is really needed for your module to be usable within the RadEditor! we really need a reliable DMX Browser like the old DnnWerk days.
I don't see how the module is useful otherwise as general Intranet page editors don't want to have to go to DMX and upload, the manually link to the files, and the file browser doesn't work. then if they upload files directly through the RadEditor Document Manager they aren't in DMX so not secure.
furthermore the security issue is huge for intranet use as there are a mixture of public and private folders so them not being able to use/see private folders is a big deal. It makes me think the module is no good anymore, even though it is great otherwise. I'm seriously thinking that the default DNN File Manager / Digital Asset Manager is the only acceptable choice now for usability, even though it lacks in other areas where your module shines.

What do you recommend as a workable and user-friendly solution?

Regards,
Tim

Tim,

I wish there was. DNN is not going in this direction at all. It is still stuck on this limited implementation of their provider. I might be able to influence things in the future, but that is just guessing. For now this is the best *I* (or any other module developer) can do. Going directly to the data (i.e. bypassing their sync logic) can lead to spectacular failures and will make the system as a whole very brittle. I definitely do not want to go there.

Peter

Peter,

I posted the original post on behalf of KCB and this is a follow-up after having purchased the module for other clients. The issue above still happens and I wanted to review some assumptions.

When I go to 'Storage Options' and set a new folder as the root it does NOT move the existing files. They remain as entries under the /Portals/0/DMX/* and still reside within the DMX module. Only NEW files from this point go to my /HTML/DMX/* folders (where HTML is my site root). As I had set the system to make monthly folders these are of course organized under /2014/[mm] etc.

* Is it expected that files uploaded previously stay in their old root? Do things function as normal this way?
* Is there a script under 'Run Scripts' to move the entries in the previous root to the new root folder? (I tried one that would "Copy existing files to a new root" and it wiped out ALL entries in my DMX instance.)
--- Many other scripts sound like they *might* cover this option but to reset my test environment and re-try them one after another is really ineffective. Especially considering the first test actually destroyed all entries in my test instance.
* The replacement DMX root folders created through setting the 'Storage Options' path lack the wide array of supporting folders the original root had. i.e. 'Log', 'Lucene', 'Menu', 'Temp', 'Thumbnails', 'Upload' and 'View Templates'. Is this expected? (They do NOT exist in my NEW root.)

The reason I'm trying to move the files into a new root is to "reboot" the synchronizing. In particular, it seems semantically incorrect to have the DMX folder exist INSIDE the DNN Portal's root for File Management.

When we consider that adding a file to DMX would add a file to /Portals/0/DMX/2014/10/file_20141015_044825_PEOh_0.resources it seems to me that we'd be "double scanning" files during the scheduled (or manually triggered) DNN File Synchronization. Additionally, it clogs the DMX integrated folder with what appears to be clutter to the end users. (i.e. I only want to see /DMX/Myfolder/File.ext NOT /DMX/2014/11/26/* -AND- /DMX/Myfolder/File.ext.)

To bring this back to the real problem we need to solve, I was thinking doing the "ideal" setup would be like so:

1. Clean up what we can to shrink the 3.5GB worth of folders and files. They are stored local to the server so access speed is only limited by the disk system.
2. Change DMX to use a 'Storage Folder' different than the default of '/Portals/0/DMX' where DNN already looks. I am under the impression the "right way" to integrate the two is to be 100% synchronized via DNN polling the DMX DLL and never seeing DMX's "working folders" inside the portal root. I figured '/HTML/DMX' would be superior to anywhere under 'Portals/0/'.
3. Do a script or something (PER ABOVE) that would "fix" the 3.5GB of files that were off the DMX root. I can manually copy the files into the NEW folder if that is the right process to do this. I just don't know the steps, or order of steps, to achieve this. **** IMPORTANT *** We absolutely cannot break the existing links. If the system creates new file IDs and breaks links, the process will break hundreds+ of links throughout the website.
4. Optionally: I've read things that might have suggested it should be a DMX root SUBFOLDER that should be synchronized to DNN's File System? i.e. /DMX/Root/ vs. /DMX/. Does this matter? If so, how can it be done under the aforementioned circumstances?

The synchronizing is so broken now that it's effectively useless. I've got several clients using it and I want to move them to best practices so I am confident that issues that remain are in fact, defects.

Please help me sort through this.

Hi Dylan,

"When I go to 'Storage Options' and set a new folder as the root it does NOT move the existing files."
This is by design. It is how DMX works. You have to move these files yourself. As soon as you change the setting DMX has no knowledge of the old setting any more.

So "Is it expected that files uploaded previously stay in their old root? Do things function as normal this way? "
Yes

"Is there a script under 'Run Scripts' to move the entries in the previous root to the new root folder?"
No

"The replacement DMX root folders created through setting the 'Storage Options' path lack the wide array of supporting folders the original root had. i.e. 'Log', 'Lucene', 'Menu', 'Temp', 'Thumbnails', 'Upload' and 'View Templates'. Is this expected?"
Yes. Those folders are "DMX System" folders that can't be moved. They don't contain nearly as much content as the others. Note you can get a Graveyard that fills, but that you can remove periodically if you wish.

To move the repository you move all the files to the new location and you change the value for the repository root in the settings. That is the correct way. Obviously you won't move the system folders. Only the year/month folders. You have to preserve the structure though from the root up as that is what DMX stores internally.

Note: DMX was not designed as a replacement for the DNN file manager. The file manager exposes files directly in the web directory and potentially allows access to those that know the url to a file. DMX copies all files into a "vat" (repository) with it's own internal structure and naming convention. Synchornizing with the root of a DNN home directory is possible but I doubt it helps with security and you end up just doubling stuff that is exposed throught the regular file manager anyway. The way synchronization was meant to be used was to reflect of some location of a company's server where they keep internal documents accessible internally through a UNC path. Synchronization allows this to be exposed on an extranet site in this way. This was the use case that prompted the development of sync folders.

Peter

Peter,

The way the root is handled (not moving files) makes good sense to me. Thanks for the confirmation.

As an FYI: When I ran the script to "move files to the new root" it actually wiped out the *entire* DMX file structure. It didn't just "forget" the files from the old root. It even lost/removed every folder and file I had created since changing the storage root.
* I found this very odd and wonder if I've found a bug in the script or if I'm naive as to how it was designed to work.

I'm glad you've confirmed my concerns that putting the DMX content into the Portal's File Manager root was going to be a form of duplication or redundancy. This was established before I became involved in managing this particular portal. That said, the documentation and defaults for DMX are probably what lead people to do this before me. (I checked documentation to learn best practices and was surprised to find it did recommend '/Portals/{PortalID}/DMX'.

I'll do another test today where I:

1. Establish test folders/files under '/Portals/0/DMX'
2. Set a new root, say '/HTML/DMX'
3. Manually copy the files from the old root to the new root
4. Test add, revise, delete on both old and new folders and files
5. Test the existing DMX file links (in HTML Content modules on the site) to confirm download links are not broken

If you have anything to add, I'd love to hear it and/or test out steps. Either way I'll let you know the outcome of this test.

Hoping file synchronization resumes and this process doesn't break the existing document structure and functionality!

Thanks, Dylan

Hi Dylan,

"When I ran the script to "move files to the new root""
This is not the correct name for the script. It should be: move the files to the current provider. What it does is help move files from S3 to Web server or vice versa. So it only has a bearing on when you have switched *providers*, not the root.

The latter approach you outline is the correct one.

Peter