Apache Allura™ / Tickets / #5697 Timeout on solr queries

#5697 Timeout on solr queries

Milestone: v1.0.0

Status: closed

Owner: Cory Johns

Labels: performance (126) sf-1 (616)

Component: General

Reviewer: Dave Brondsema

Updated: 2015-08-20

Created: 2013-01-28

Creator: Dave Brondsema

Private: No

Some solr queries can be really slow. We should have a configurable timeout on our solr queries so the original web request stops after several seconds and doesn't jam things up.

Make sure this doesn't negatively impact add/remove artifact tasks that could be huge and take a long time (which is ok). Do we have other solr read queries that don't originate from web requests and should be given long timeouts?

Discussion

Dave Brondsema - 2013-01-28

size: --> 1
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Cory Johns - 2013-01-28

status: open --> in-progress

assigned_to: Cory Johns
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Cory Johns - 2013-01-28

allura:cj/5697

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Cory Johns - 2013-01-28

status: in-progress --> code-review
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Dave Brondsema - 2013-01-29

status: code-review --> in-progress

qa: Dave Brondsema
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Dave Brondsema - 2013-01-29

One line uses with_timeout which isn't right.

Overall, it's not pretty, but that's more pysolr's fault than ours. Could search_artifact use the search() method too, so the if stmt doesn't have to be duplicated?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Cory Johns - 2013-01-29

It could not, as search() is wrapped with @try_solr.

As for making it prettier, I could make our subclass redefine search() to accept a timeout instead of requiring it at __init__ by temporarily setting self.timeout before calling super, but I was mildly concerned about potential concurrency issues.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Cory Johns - 2013-01-29

Alternatively / additionally, we could make the default value be the common case (short timeout) so that we don't have to manually specify it every time, but I went with default as long to be safer if I missed any cases (scripts).

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Cory Johns - 2013-01-29

with_timeout fix pushed

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Cory Johns - 2013-01-29

status: in-progress --> code-review
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Cory Johns - 2013-01-29

Consolidation of Solr error handling added.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Dave Brondsema - 2013-01-29

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Dave Brondsema - 2013-01-29

status: code-review --> closed
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Apache Allura™

Forge software for hosting software projects

Milestone

Searches

Help

#5697 Timeout on solr queries

Discussion