Indexing: Finder and DEVONthink Relationship

Hi. I think some of these questions have been answered, but others I haven’t been able to confirm anywhere. This is how I understand the relationship between the DT database and Finder for an indexed folder (with sub-folders).

(1) Change the name of a file in DT, update indexed items, and the change is reflected in Finder. Follow the same steps in Finder and the change is reflected in DT.

(2) Delete a file or folder in DT, empty the trash (deleting “files and folders”), update indexed items, and the change is reflected in Finder.

(3) Change the name of a group in DT, update indexed items, and the change is NOT reflected in Finder.

(4) Replicate or duplicate a file into another group, update indexed items, and the change is NOT reflected in Finder.

(5) Create a smart folder in Finder, update indexed items, and it will appear in DT, but the contents won’t.

(6) Create a smart group in DT, update indexed items, and it will NOT appear in Finder.

Did I get everything right? Please let me know if I am confused about something here.

Ideally, I would be able to mirror the Finder contents in DT, but it looks like a certain amount of dissonance is inevitable at this point. DT groups are not Finder folders, of course, but indexing Finder folders does create DT groups, so the expected behavior (from my perspective) is to have basic folders and groups mirror one another. The smart folder thing kind of makes sense, because the smart folder content might be constantly in flux, and pulling in data from all over the place.

The replicants not showing up in Finder kind of makes sense as well, because OSX doesn’t have replicants, but I expected duplicates to show up. None of these features are huge problems, but they aren’t necessarily desirable behaviors for my particular use case either. In particular, the inability to change group names has caused a bit of trouble for me. I could, theoretically, get around some issues by using tags. However, tags are useless in DTTG, and it would become quite difficult to navigate many thousands of files on the iPad without some structure and/or more granular searching.

Looks accurate to me.

An important thing to keep in mind is that for each indexed item DEVONthink knows the path from the root of your drive to the location of that item in the file system, and the only way to change that path is to import the item (“move into database”) then export it to another location (“move to external folder”). Otherwise, you can move the indexed item around anywhere in the database you choose. The structure (hierarchy) of things in the database only reflects the external world in the file system if you never change names, locations, tags, or other attributes of the documents – otherwise you can make the internal structure as different as you wish.

Though personally I’m with you on keeping group and folder names in sync – I can see a case for users who want their internal group names to vary from the folders. Replicants not showing up in the file system is inevitable because, unlike aliases or symlinks, replicants are not “real” objects – they are representations generated on the fly when you view the group where the replicant lives. Just like the content of smart groups is generated on the fly when you look inside the smart group.

Many readers have been confused by indexing because they anticipate that using DEVONthink is like using a file system browser like Finder or PathFinder. It is not like that at all, though. The database is a structure you create with the freedom to make it wholly different than the file system. That’s why many users have benefited from never maintaining structure in the file system, just dumping all their data into massive folders, indexing those folders, and using DEVONthink’s power to create and evolve the structure. It’s a good approach.

Thanks a lot. I could definitely put up with the dissonance, and maybe even benefit from it, if DTTG could handle tags. It pretty much ignores them and I have adjusted my OSX workflow to take advantage of groups, which DTTG does well.

My preferred method of working is actually to only have a handful of folders / groups (I’ve worked for years with just one), but in order to make that function well, I need to have strong support for tags and/or granular searching. Searching on OSX is fine (more on that below), but on DTTG it is like pulling teeth, and I try to avoid it.

One of the tough things for me in transitioning from Evernote into DT was my inability to stay organized in a single group. I started that way, but for a long time, I couldn’t quite put my finger on the problem I was having. Eventually, I figured out that it has to do with how the searching is done – in Evernote, you use the search field to filter what you see in the center column and saved searches essentially function as “groups” or “smart folders.” I like that a lot, especially on the iPad.

DT’s searching has its strengths, to be sure, but it doesn’t work as effortless for me. In order to employ multiple search parameters (perhaps all files with X in the name and Y in the content), I have to open a separate window and go into the advanced settings. It would be nice for me if we had an advanced search grammar we could just put into the search field as Evernote does (intitle:X Y). It’s not that one way is right or wrong, but I have discovered that I really enjoy working with a search field that filters my results and can be easily manipulated to return different results. BBEdit is also a joy to use, except for its inability to search on file names. Something like either of these would be nice with DEVONthink: (group:X name:Y content:Z)?

All of this is to say that some of the problems I am having with indexing stem from a lack of support for other features in DTTG (I need to use more groups, which don’t mirror what is in Finder, so I end up with a mess in Finder). Perhaps DTTG2 will alleviate some of the problems by enabling us to search, sort, and navigate with tags. I don’t know, but I think it will probably be best for me to rethink my strategy.

The only thing I would add is that for your item #1 and #2, it is not necessary to manually update indexed items. DEVONthink will reflect these changes in the Finder automatically.

Thanks Greg. I’ll watch it more closely to see exactly when it updates.

(7) Modify a tag name in DT, update indexed items, and the change will NOT be reflected in Finder.

In my opinion, while there may well be reasons not to mirror some aspects of a database in Finder, having different names for tags and groups makes for quite a mess. I wonder why file names are mirrored, but almost nothing else is. It’s not “wrong,” but it seems to make indexing that much more difficult to use, because the behavior is unexpected and counter-intuitive. True, the current system allows for creativity in the DT database, because you don’t have to worry about mucking things up in Finder. That’s nice, but what about people who are looking to mirror there things, a common use case I see on this forum?

Why mirror things when I could just import it all into DT? Well, in my case, I have most of my files in SpiderOak and everything is accessible through that from any computer or mobile device. Non-sensitive items go in Dropbox so that I can write on the Mac with Scrivener and use the iPad with notesy. In addition, my DT database is too large to sync everything with my iPad, but because everything is also in the cloud, I can still access files if needed using the iPad.

No, the change is reflected. There are occasional delays with OS X updating Mavericks/Yosemite tags – especially if several files and/or tags are involved. But the update does occur. (Delays are an OS X problem, not caused by DEVONthink.)

(Just tested this 23 seconds ago and it works )

No luck here. I just re-tested it and it’s been a few minutes. I updated indexed items, and I closed and re-opened the app for good measure. In this case, 96 files with a single tag are involved. I’ll fiddle around some more with it, but I wonder what the problem could be. Basically, I have indexed a single folder and I am testing with tags applied to files within it. I’m running DTPO 2.8.4 on Yosemite 10.10.2.

[EDIT:] 24 minutes later it still hasn’t changed. I don’t see it happening today. In the meantime, I am upgrading Yosemite to 10.10.3. I didn’t realize I had fallen behind.

Don’t know if you have done this yet, or even if it will make a difference, but it might be worthwhile to turn off OpenMeta tags support in DEVONthink. I believe I saw improved tag reliability between the Finder when I set this preference.

Quit DEVONthink, and then enter this command in Terminal:

defaults write com.devon-technologies.thinkpro2 DisableOpenMeta -bool TRUE

Thanks Greg! I’ll give that a try and report back. At the moment, I am in the midst of reorganizing stuff. I’ve been experimenting with indexing for a while now (years?) and I think I’m going to have to finally concede that I may be better off importing instead for my use case.

Problems with Tags in OS X are usually related to Spotlight’s performance and the health of its index, not OpenMeta. An mdProcess could be stalled on a single file as it attempts to index it. The index could have corruptions the cause a loop in the lookup. …

Remember that there is a index just like DEVONthink has. An occasional Spotlight index rebuild is not a bad idea.
This is my favorite page of layman’s info on Spotlight: thexlab.com/faqs/stopspotlightindex.html

I hope it’s of some use (and it’s still very pertinent in general).

That is a great link. Thanks so much! In this case, I had to take my computer to the shop a few days ago, and everything is a relatively fresh install, so I don’t think the index is the issue this time. But, I’ll give it a try anyhow.