Just a little tip which seems obvious now that I’ve done it, but may save some hassle for someone in the future. Combining reference software and DTPO is a bit of a constant topic here and on the web. For me the crux was to not duplicate PDFs. If I’ve done a load of annotations etc on a bunch of PDFs, the last thing I want is to have non-annotated copies of them in a reference manager (not to mention them taking up drive space). I just want all my research etc in one place, which for me is DTPO. BUT I also wanted to find a way of my reference software to automatically generate citations from PDFs imported into them, which is one of the very cool things about them.
The solution is just to set Mendeley to watch where DTPO stores its PDFs within DTPO. This can be found in the database (yourdatabase.dtBase2/Files.noindex/pdf). And that’s it. Now every time you import a PDF into DTPO, Mendeley will automatically scan it and generate a record of it in Mendeley. This technique will also work (I would assume) for those other reference managers that can watch folders. You could of course reverse this, by setting DTPO to index the folder that contains the PDFs you import directly into Mendeley (or Bookends, Sente etc), but personally I’d rather have all my stuff in DTPO.
Configuring an app to access the internals of your database packages, while feasible, is not without risk. For example, you have no idea that a future release of that app would not damage your database. Just be aware of the risk.
No – I mean a future release of Mendeley. You can be pretty certain that Mendeley is not testing their software to ensure it works with the internal structure of a DEVONthink database package.
Probably not, as DTPO is using Apple’s code for PDFs.
But in any case we discourage approaches that directly address files stored within a database’s Files.noindex folder, as there are 1) increased possibilities for database damage and 2) probable failure in reliably accessing a specific file, as DEVONthink dynamically manages file storage within the database for performance reasons, so that file locations may change over time.
Thanks for your comments. I was interested to read how DTPO “dynamically manages file storage within the database for performance reasons”. So which other places would DTPO store PDFs?
It will store PDFs within a location it chooses inside the .dtBase2 package. Or – it indexes the PDF from the location your choose outside the database. No other places.
Yes. If DEVONthink and another program need to access the same files, the correct solution is to index those files in DEVONthink.
Personally, I don’t use Mendeley, but I index my Bookends attachments folder in DEVONthink. I also index my nvALT folder (and access that same folder in DEVONthink, Curio, Simplenote, and nvALT). I index my Notebooks folder in DEVONthink. I index my Ulysses “external” folders in DEVONthink. And so on. I think the two common reasons to do this is the desire to access files in a reference manager and DEVONthink, and the need to access files in DEVONthink that are also used in iOS versions of macOS applications.
Whenever two programs access the same files there is the possibility that something could go wrong. Most commonly this happens when both programs try to change the same file at the same time. But that’s a risk that is very minimal, and pretty much under the user’s control – unless one makes a habit of trying to simultaneously use two editors for the same file. Not a big risk for most users.
So, I understand that DTPO stores PDFs in the database when you import them, but where apart from yourdatabase.dtBase2/Files.noindex/pdf might it place them when it “dynamically manages file storage”?
What Bill means is that DEVONthink might move your PDFs from one subfolder to another within the database – i.e., dynamically – as it manages your database behind the scenes. He didn’t mean that DEVONthink puts your PDFs (or any file) somewhere else on your machine.
Thanks, korm. Because the subfolder location of any file stored within the database may change, an attempt to link to that file from outside will be unreliable.
Many thanks for the informative replies. Looking back at the notes detailing my thinking process at the time, my decision to take the path I did was dictated by some bad software choices. I had decided to use Papers3 as my reference manager (it came as part of a package that also gave me DEVONagent), but this was so opaque that I was unable to index it. Plus one of my big must haves was to have a fully searchable database, and when I looked on this forum for OCRing indexed files, it seemed rather more hassle than it was worth, certainly compared to just importing it, although if I had the info I have now, I think for those that needed OCRing and a reference, I would have imported them, OCRed them, and then exported the whole file to the indexed folder of Mendeley, Bookends whatever.
So at this point I decided to just import everything into DTPO, which also meant all my files were in one place, and I pretty much gave up on the idea of syncing the two, and decided to just leave the idea for a bit and keep on the lookout for a solution. I then ditched Papers3 because it got into the habit of spontaneously deleting references (you had one job P3, ONE JOB!), which is when I looked again at software choices, and realised I really don’t need a ‘bells and whistles’ ref manager, or even CWYW. I just needed something that could generate a full reference that I could then copy and paste into my PhD, which is when I hit upon the solution above. I literally use Mendeley for nothing else except for it to watch a folder and then generate a ref, so no conflict issues will arise. The only real sod is DTPO ‘dynamically managing storage’ so that PDFs could be moved to (it seems) anywhere in the database. But that is a real sod.
It’s clear that it’s not a very good solution, but as having a fully searchable database was my top priority, it’s the one I’m going to have to stick with I think, especially as I have expended a lot of effort in categorising all my imported files. Again, if this was early days, I could go back and start again, but I feel I’ve gone a little too far down this road now, and to be honest, I’ve spent far too much of my time on processes rather than doing my actual PhD – at some point you have to draw a line and say enough is enough. What effort i have left regarding this will be spent on trying to get DEVONagent to do what I want, but even that will have to wait for a bit!
So am I correct that DTPO does not OCR indexed documents (i.e. my bookends folder) and you have to import them? Not running academic references through OCR would mean missing out on much of the AI capability. And from the above comments pointing bookends to a folder inside a DTPO database doesn’t look like the way forward.
Suddenly unsure of the best way to combine the two!
DTPO can perform OCR on an Indexed file, but with the result that a copy of the original file is Import-captured to DEVONthink. In DTPO Preferences > OCR, you can check an option to delete the original file following OCR.
Then, if the new searchable PDF is located in an Indexed group (presumably your citation manager database folder), you can select one or multiple non-Indexed (Imported) items in that group, Control-click and choose Move to External Folder. The PDFs will be moved from the DEVONthink database out to the designated Finder folder, then Index-captured back into DEVONthink.
Many thanks Bill.
So, I create an indexed group in DTPO and run the OCR on them (which creates the imported version), then follow the instruction below, moving the searchable versions back out to the indexed folder?
I haven’t really looked into scripts closely yet but I’m guessing that could be automated?
Thanks again,