# EPAM LifeSciences Open Source Products #

## Introduction ##

This repository includes indigo based solution for searching molecules. Indexing molecules is done via Lucene/SOLR.

## Demo project ##

Before going into technical details, we mention demo project which tries to be close to minimal, yet illustrative example of using our solution. At the moment it is simple set of example queries REST-endpoints with web ui generated by swagger for each of these endpoints. The code for demo project resides in **demo** folder. To run it you have to have [docker-compose 1.6.0+](https://docs.docker.com/compose/) installed. Store molecular data you want to index in demo/chem-data (available formats include smiles and sdf files). After that you can get demo project up and running with just one command: from **demo/docker** directory do
```bash
sudo docker-compose up --force-recreate
```
After some time you will get UI available at http://localhost:8080/webservice/restapi/swagger-ui.html . When doing test queries do not forget to change limit to something greater than zero. Example of query to **findExact**:
```json
{
  "limit": 5,
  "offset": 0,
  "structure": "c1ccccc1"
}
```
You can see to **demo/docker/docker-compose.xml** to find out which ports are open and other details.

## Architecture ##

This solution consists of two parts: plugin for SOLR (aka cartridge) and query language for building queries to SOLR augmented with cartridge.

### Building query plugin for SOLR using maven ###

From the root of the project directory do

```bash
mvn install
mvn -pl chem-query-plugin package assembly:single
```

Now copy chem quety plugin to your solr plugins path:
```bash
cp dist/chem-query-plugin-<version>.jar <your-solr-path>/plugin
```

Now you can do queries, but you will need a client.


### Creating client ###

Clients for chem query solr plugin are done in typesafe way (we build a small type-safe DSL), so they are parametrized by schema of your solr collection(s).
At this moment DSL does support only string fields (including multiple) and molecular fields. Of course, your schema can contain any fields supported by solr,
but only mentioned subset will be representable in our DSL. 

To make queries to SOLR you first should generate java class for your schema and then make queries with DSL.

Let us see how this works. Suppose you have a following schema:

```xml
<schema name="moldocs" version="1.5">
    <fieldtype name="long" class="solr.TrieLongField" precisionStep="0" positionIncrementGap="0"/>
    <fieldtype name="string" class="solr.StrField" indexed="true"/>
    <fieldtype name="strings" class="solr.StrField" multiValued="true"/>
    <!--Serialized molecule -->
    <fieldtype name="T_serMol" class="solr.BinaryField" indexed="false" stored="true"/>
    <fieldtype name="T_molId" class="solr.TextField" indexed="false" stored="true"/>
    <fieldtype name="T_fingerprint" class="solr.TextField" multiValued="true" indexed="true" stored="true">
        <similarity class="com.epam.indigolucene.solrext.ChemSimilarity"/>>
    </fieldtype>
    <fieldtype name="T_molsim" class="solr.TextField" indexed="true" stored="true" />
    <fieldtype name="T_docUrl" class="solr.TextField" indexed="false" stored="true"/>
    <fieldtype name="T_docId" class="solr.TextField" indexed="true" stored="true"/>

    <similarity class="solr.SchemaSimilarityFactory"/>

    <!--<field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" />-->
    <!--<uniqueKey>id</uniqueKey>-->
    <field name="_version_" type="long" indexed="true" stored="true"/>
    <field name="mol" type="T_serMol"/>
    <field name="molId" type="T_molId"/>
    <field name="mol_fingerprint" type="T_fingerprint"  termVectors="true" termPositions="true" termOffsets="true"/>
</schema>
```
With com.epam.indigolucene.schemagenerator.JavaSolrSchemaRepresentationGenerator class from schema generator module you can generate all necessary classes
You need to provide four parameters to its method 'main': preferred name of schema class, path to schema.xml, path to sources folder and the name of package you want to put genetrated files to.
If you use maven, this can be done on sources generation phase like this:
```xml
<plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>exec-maven-plugin</artifactId>
                <version>1.1.1</version>
                <executions>
                    <execution>
                        <phase>generate-sources</phase>
                        <goals>
                            <goal>java</goal>
                        </goals>
                        <configuration>
                            <mainClass>com.epam.indigolucene.schemagenerator.JavaSolrSchemaRepresentationGenerator</mainClass>
                            <arguments>
                                <argument>TestSchema</argument>
                                <argument>${basedir}/../conf/schema.xml</argument>
                                <argument>${basedir}/src/main/java/</argument>
                                <argument>com.epam.indigolucene.service.generated</argument>
                            </arguments>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
```
Also, you have initialize connection factory with some connector class:
```java
SolrConnectionFactory.init(SolrConnection5.class)
```
After that you can use newly generated classes to do queries like this:
```java
TestSchema.collection(<url>, <core_name>).find().filter(TestSchema.MOL.hasSubstructure(<indigo_object_of_structure>))
                                                                      .limit(5)
                                                                      .offset(0).processWith(<response processor>);
```
or
```java
TestSchema.collection(<url>, <core_name>).find().filter(TestSchema.MOL.unsafeHasSubstructure(<string_structure_representation>))
                                                                      .limit(5)
                                                                      .offset(0).processWith(<response processor>);
```
Also here is an example of adding documents to SOLR:
```java
try (SolrUploadStream<TestSchema> uStream = TestSchema.collection(<url>, <core_name>).uploadStream()) {
    TestSchemaDocument emptyDocument = TestSchema.createEmptyDocument();
    emptyDocument.setMol(<string_structure_representation>);            
    uStream.addDocument(emptyDocument);
}
```

## Running Tests ## 
### integration tests ###
If You have a desire to run some tests(to make sure that functionality you want to use is working properly) just run a command presented below in **integration-tests** directory
```bash
mvn test
```

For more specific approach(if You want to run a single test class), use that command instead: 
 ```text
 mvn -Dtest=<Your_test_classname> test
 ```
 
 And for running a specific test method inside a test class use: 
 ```text
 mvn -Dtest=<Your_test_classname>#<class_test_method> test
 ```
 ### benchmark tests ###
 In case if You want to run benchmark tests, first head to 'benchmark-tests' folder,  then run 
 ```text
  mvn clean install
 ```
 Afterwards 
 
 ```text
 java -jar target/benchmarks.jar
 ```
 
 <b>NB</b>: It is highly recommended to close all other apps for more accurate test results