From owner-chemistry@ccl.net Wed Nov 12 11:18:01 2014 From: "Michael Sluydts michael.sluydts-,-ugent.be" To: CCL Subject: CCL: reaction database summary Message-Id: <-50741-141112110739-11391-1R4IqcLSQ7t8nU1vHTpwLA]=[server.ccl.net> X-Original-From: Michael Sluydts Content-Type: multipart/alternative; boundary="------------000305040400000105040909" Date: Wed, 12 Nov 2014 17:07:16 +0100 MIME-Version: 1.0 Sent to CCL by: Michael Sluydts [michael.sluydts---ugent.be] This is a multi-part message in MIME format. --------------000305040400000105040909 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hello Jozsef, To respond to the comments on my email: -It may indeed be more practical to just do everything in Python -Python can however also interact with MySQL and it may be more efficient to preprocess most things and store them in the DB -You can also call shell commands from PHP (including python) -For the database, while postgre will do the job (and there's a certain element of taste involved) I do think MySQL (free) and Oracle (paid) would be the most common choices (microsoft SQL is used a lot too but I wouldn't recommend Windows servers...). For my high-throughput system I always use Python for operations in the computing environment (and running the calculations) and PHP/JS/jQuery for the web interface. The interaction between Python and the database I actually do through a JSON API I programmed in PHP (both PHP and Python let you easily encode arrays and objects into the JSON string format making it easy to interact). Best regards, Michael Sluydts Jozsef Csontos jcsontos.lists-*-gmail.com schreef op 12/11/2014 14:36: > Dear Members, > > As I promised here comes the summary of answers for my inquiry about > the implementation of a chemical reaction database. > > To sum up, it seems that the setup: postgreSQL DB + rdkit + web2py is > viable for implementing a reaction database. > > _Detailed responses follow:_ > > /If you're developing a Python-based web interface with the > requirement that the backend libraries be open source, you might also > consider the Django package. It introduces a model-view architecture > that gives you good abstraction and reusability, and you may find > development using it to be faster than with other packages. It's > fully compatible with Postgre, along with many other database servers. // > //// > //There's also a lot of modular packages already available for it > (again open source), including user management (included in the main > package), user registration, etc. For these reasons, you may find > that Django significantly speeds up your development process.// > //// > //Best,// > //James > / > Thank you James, I was hesitating whether django or web2py is the > right framework and based on my very limited experience I selected > web2py because of its more gentle learning curve. > / > //Will you effectively only have starting material and product or will > you be implementing some kind of atom mapping?/ > / > / > /Cheers/ > / > / > /Chris// > / > No, Chris, nothing fancy like that. > I just want to be able to insert good, old published reactions with > their associated properties like who, where, and when published it and > some physicochemical properties of them. > > /Hey,// > //// > //I may be a bit late in the discussion but I'd recommend using > technologies which were designed for websites. The concept seems > fairly simple so it would not be too hard to develop it from scratch.// > //// > //I recommend using PHP rather than Python for the code, a MySQL > database and for ease of use but a modern layout something based on > //http://www.initializr.com///using the twitter bootstrap layout > (responds to screen resolution). How to use the layout can be found > here: //http://getbootstrap.com/components///.// > //// > //Depending on the needs and ideas I might be able to offer some > help/infrastructure if needed. Feel free to send me a direct email if > so.// > //// > //// > //Best regards,// > //// > //Michael Sluydts/ > / > /Thank you Michael, > however, I really need python because I'm planning to do some > manipulation of the physicochemical data using python libraries. > / > / > /Jozsef,/ > / > / > /I would also recommend MOLDB5 > (//http://merian.pch.univie.ac.at/~nhaider/cheminf/moldb5.html > //). > It has a version (MOLDB5R) that handles reactions./ > / > / > /I used this program to build a very flexible inventory and data > repository. All aspects are open source. It uses MySQL, Perl, and a > Java structure search tool./ > / > / > /Soren/ > / > /Thank you, Soren, > I checked it out and it seems that it is a really nice project it > might be a good plan B for me. > / > / > /Dear Jozsef,// > ////// > //Are you trying to create an open source version of Reaxys (formerly, > Beilstein)?// > //// > //http://www.elsevier.com/online-tools/reaxys// > //// > ////Karen// > / > No, Karen, I don't have that in my mind. > > /I think there's a huge demand for this kind of database. One catch > might be that elementary reactions don't have obvious citations./ > // > / > / > // > /But I know from Open Babel and Avogadro, that there have been many > requests for an "open data" reaction set. (Indeed, one idea for > Avogadro would rely on such a set of reactions - highlight functional > groups from the SMARTS and then show potential reactions to students.. > essentially marrying step-by-step synthetic schemes with molecular > visualization.)/ > // > / > / > // > /You may want to consider reaction SMILES or SMIRKS, and allow for > side-reactions, catalysts, reaction-conditions, etc./ > // > / > / > // > /I think what you're proposing should be fairly robust, but I'd spend > a lot of time doing load-testing and security testing. There are a > *lot* of automated scans out there, and DDOS attempts. On the plus > side, there are free services like CloudFlare that can help defend > against some of these./ > // > / > / > // > /Please let me know if there's anything I can do to help. I've wanted > something like this but haven't had the time or energy to devote to > getting it off the ground. I'm thrilled there's someone else willing > to get started./ > // > / > / > // > /Thanks and best regards,/ > // > /-Geoff/ > // > / > / > // > /P.S. You should also look into the OSRA package that should be able > to do some image recognition, and the set of reactions in the US > Patent database (in ChemDraw format)./ > > Geoff, thanks you for your suggestions, > I think that both openbabel and avogadro are really great projects. We > already use the openbabel python module and I've also done some > in-house avogadro "development" for a totally different project. In > the near future I'll be back with that :) > > Finally, thank you for everyone and all the best, > > Jozsef > > --------------000305040400000105040909 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit 
Hello Jozsef,

To respond to the comments on my email:
-It may indeed be more practical to just do everything in Python
-Python can however also interact with MySQL and it may be more 
efficient to preprocess most things and store them in the DB
-You can also call shell commands from PHP (including python)
-For the database, while postgre will do the job (and there's a certain element of taste involved) I do think MySQL (free) and Oracle (paid) would be the most common choices (microsoft SQL is used a lot too but I wouldn't recommend Windows servers...).

 For my high-throughput system I always use Python for operations in the computing environment (and running the calculations) and PHP/JS/jQuery for the web interface. The interaction between Python and the database I actually do through a JSON API I programmed in PHP (both PHP and Python let you easily encode arrays and objects into the JSON string format making it easy to interact).


Best regards,

Michael Sluydts
Jozsef Csontos jcsontos.lists-*-gmail.com schreef op 12/11/2014 14:36:
Dear Members,

As I promised here comes the summary of answers for my inquiry about the implementation of a chemical reaction database.

To sum up, it seems that the setup: postgreSQL DB + rdkit + web2py is viable for implementing a reaction database.

Detailed responses follow: 

If you're developing a Python-based web interface with the requirement that the backend libraries be open source, you might also consider the Django package.  It introduces a model-view architecture that gives you good abstraction and reusability, and you may find development using it to be faster than with other packages.  It's fully compatible with Postgre, along with many other database servers. 

There's also a lot of modular packages already available for it (again open source), including user management (included in the main package), user registration, etc.  For these reasons, you may find that Django significantly speeds up your development process.
Best,
 James

Thank you James, I was hesitating whether django or web2py is the right framework and based on my very limited experience I selected web2py because of its more gentle learning curve.

Will you effectively only have starting material and product or will you be implementing some kind of atom mapping?

Cheers

Chris

No, Chris, nothing fancy like that.
I just want to be able to insert good, old published reactions with their associated properties like who, where, and when published it and some physicochemical properties of them.

Hey,
I may be a bit late in the discussion but I'd recommend using technologies which were designed for websites. The concept seems fairly simple so it would not be too hard to develop it from scratch.
I recommend using PHP rather than Python for the code, a MySQL database and for ease of use but a modern layout something based on http://www.initializr.com/ using the twitter bootstrap layout (responds to screen resolution). How to use the layout can be found here: http://getbootstrap.com/components/ .
Depending on the needs and ideas I might be able to offer some help/infrastructure if needed. Feel free to send me a direct email if so.

Best regards,
Michael Sluydts

Thank you Michael,
however, I really need python because I'm planning to do some manipulation of the physicochemical data using python libraries.


Jozsef,

I would also recommend MOLDB5 (http://merian.pch.univie.ac.at/~nhaider/cheminf/moldb5.html).  It has a version (MOLDB5R) that handles reactions.

I used this program to build a very flexible inventory and data repository.  All aspects are open source.  It uses MySQL, Perl, and a Java structure search tool.

Soren

Thank you, Soren,
I checked it out and it seems that it is a really nice project it might be a good plan B for me.


Dear Jozsef,
 
Are you trying to create an open source version of Reaxys (formerly, Beilstein)?
http://www.elsevier.com/online-tools/reaxys

 Karen

No, Karen, I don't have that in my mind.

I think there's a huge demand for this kind of database. One catch might be that elementary reactions don't have obvious citations.

But I know from Open Babel and Avogadro, that there have been many requests for an "open data" reaction set. (Indeed, one idea for Avogadro would rely on such a set of reactions - highlight functional groups from the SMARTS and then show potential reactions to students.. essentially marrying step-by-step synthetic schemes with molecular visualization.)

You may want to consider reaction SMILES or SMIRKS, and allow for side-reactions, catalysts, reaction-conditions, etc.

I think what you're proposing should be fairly robust, but I'd spend a lot of time doing load-testing and security testing. There are a *lot* of automated scans out there, and DDOS attempts. On the plus side, there are free services like CloudFlare that can help defend against some of these.

Please let me know if there's anything I can do to help. I've wanted something like this but haven't had the time or energy to devote to getting it off the ground. I'm thrilled there's someone else willing to get started.

Thanks and best regards,
-Geoff

P.S. You should also look into the OSRA package that should be able to do some image recognition, and the set of reactions in the US Patent database (in ChemDraw format).

Geoff, thanks you for your suggestions,
I think that both openbabel and avogadro are really great projects. We already use the openbabel python module and I've also done some in-house avogadro "development" for a totally different project. In the near future I'll be back with that :)

Finally, thank you for everyone and all the best,

Jozsef



--------------000305040400000105040909--