#7907 Use standardized solr installation

v1.4.0
closed
Igor Bondarenko
sf-4 (350) 42cc (432)
General
Dave Brondsema
2015-12-14
2015-06-29
Dave Brondsema
No

We have a hacky setup for solr (both step-by-step and docker install):

  • we wget solr, instead of using a system package
  • we copy our schema into the solr example dir
  • we run start.jar directly instead of a service

For docker, we should be able to use a solr image that somebody else has set up. For the step-by-step we should use system packages and perhaps borrow ideas from the solr container's Dockerfile.

Discussion

  • Dave Brondsema

    Dave Brondsema - 2015-10-05
    • labels: --> sf-4, sf-current
     

  • Igor Bondarenko - 2015-11-05
    • labels: sf-4, sf-current --> sf-4, sf-current, 42cc
    • status: open --> in-progress
    • assigned_to: Igor Bondarenko
     

  • Igor Bondarenko - 2015-11-11

    There are official docker image for solr now.

    Unfortunately it only supports solr 5.3 (we use 4.2.1 currently).

    There another popular image but it is deprecated in favor of official image and the earliest version it supports is 4.10.

    Should we update to solr 5.3? AFAIK we're not using any fancy solr options, only schema.xml to describe solr schema, format of which did not changed between versions.

    List of major changes can be found here.

    Of course we can create own solr image, but keeping up to date with versions would be better, I think.

    And solr 5 introduces some "Ease Of Use/Operational Support" improvements which can be useful for this ticket specifically :)

    Any concerns I'm missing?

     

    • Dave Brondsema

      Dave Brondsema - 2015-11-11

      I think going to 5 would be fine for our recommended and documented installation process - it has been out for most of the year. As long as its still possible for people to run Allura on solr 4 (or older). Which I think it would, since as you say, we aren't doing anything fancy. And I absolutely agree using an official image would be best. Kind of the point of this ticket :)

      FYI, you may run into an issue with the "cat" field. We were testing internally on Solr 4.10.2 with the allura core xml files and got an error mentioning <str name="field">cat</str> and it says there is no field cat. We removed several fields from the xml file to get it to work.

      We also have config precendent (solr.use_new_types) for supporting multiple versions, if we do have code that needs to work differently on different versions.

       

  • Igor Bondarenko - 2015-11-26
    • status: in-progress --> review
     

  • Igor Bondarenko - 2015-11-26

    Closed #860. ib/7907

    For docker setup need to manually created and chmod 777 directory for solr data (see updated docs), I know it's a bit ugly, but didn't find a better way.

    For local setup we still using manual download of solr image 'cause package for ubuntu is old (~3.6). But it is better than before, since solr now has script to install itself as a service.

     

  • Dave Brondsema

    Dave Brondsema - 2015-12-04

    I've cherry-picked [3ea4c9] for the node installation, since that's a good fix to get out before our release. I haven't had a change to closely review all the rest of this ticket yet.

     

  • Dave Brondsema

    Dave Brondsema - 2015-12-08
    • status: review --> in-progress
    • Reviewer: Dave Brondsema
     

  • Dave Brondsema

    Dave Brondsema - 2015-12-08

    Do we need all the language stuff? I don't think we really use it, and it's a lot of files, 196K. Other unnecessary stuff like currency and fields like admin_subscribed etc might be nice to clean up if we wanted. But we don't need to worry about make it as minimal as possible either, just seemed like lang files were quite a lot to add.

    docker-compose says:

    Warning: the mapping "solr_config/allura:/opt/solr/server/solr/allura" in the volumes config for service "solr" is ambiguous. In a future version of Docker, it will designate a "named" volume (see https://github.com/docker/docker/pull/14242). To prevent unexpected behaviour, change it to "./solr_config/allura:/opt/solr/server/solr/allura"

    The "Mention chmod in the docs" commit can probably be changed to go into the fairly new init-docker-dev.sh

     

  • Igor Bondarenko - 2015-12-09

    I'm not sure about lang files. I added it because we have a lot of language-specific field types defined in our schema.xml and I tried to reuse old schema as much as posible.

    For example:

        <!-- Russian -->
    <fieldType name="text_ru" class="solr.TextField" positionIncrementGap="100">
      <analyzer>
        <tokenizer class="solr.StandardTokenizerFactory"/>
        <filter class="solr.LowerCaseFilterFactory"/>
        <filter class="solr.StopFilterFactory" ignoreCase="true" words="lang/stopwords_ru.txt" format="snowball" />
        <filter class="solr.SnowballPorterFilterFactory" language="Russian"/>
        <!-- less aggressive: <filter class="solr.RussianLightStemFilterFactory"/> -->
      </analyzer>
    </fieldType>

    I'm not sure if we're using them somewhere... Just checked now. In schema.xml we're using only text_general and text_ws, so I'll try to get rid of others and delete lang files.

    I'll fix warning and chmod too. I used older docker back then, so didn't see that.

     

  • Igor Bondarenko - 2015-12-09

    Closed #873. ib/7907a

     

  • Igor Bondarenko - 2015-12-09
    • status: in-progress --> review
     

  • Dave Brondsema

    Dave Brondsema - 2015-12-09
    • status: review --> closed
     

  • Dave Brondsema

    Dave Brondsema - 2015-12-14
    • labels: sf-4, sf-current, 42cc --> sf-4, 42cc
     

  • Dave Brondsema

    Dave Brondsema - 2016-04-11
    • Milestone: unreleased --> v1.4.0
     

Log in to post a comment.