Multiple indexes/cores (aka solr-215) functional value?

The idea of the multiple core/indexes feature has been
discussed in many
threads and it seems/seemed it has/had some functional
value; how do we
ensure this value is generic enough for the issue (&
patch) to ever be
solved? More importantly, is there an issue in the first
place?

For all use cases where it might make sense to use multiple
indexes, the
common (as in community and commiters) wisdom is that one
index is enough
and its schema will accommodate any functional need; from
storing documents
that have no field in common to documents in different
languages or
documents with varying publication workflows or lifetime,
etc..., one index
(schema/config/core) - fits the functional bill. In very
rare hard cases,
you might have to deploy multiple web applications.

If having only one index is never a problem, the project has
no need for a
patch that introduces multiple indexes. Thus, if the
solr-215 issue is a
non-issue, what is the process to close it and mark it as
'invalid'?
It would be of great community value to state that thinking
about using
multiple indexes is a mistake within the Solr project scope;
use Lucene or
else if this is what you need.

Is/should the single index best practice the sole one the
Solr community
needs ?
I should have asked this earlier, still learning...

-- 
View this message in context: http:/
/www.nabble.com/Multiple-indexes-cores-%28aka-solr-215%29-fu
nctional-value--tf3927076.html#a11137029
Sent from the Solr - Dev mailing list archive at
Nabble.com.

On 6/15/07, Henrib <hbiestrogmail.com> wrote:
> The idea of the multiple core/indexes feature has been
discussed in many
> threads and it seems/seemed it has/had some functional
value; how do we
> ensure this value is generic enough for the issue
(& patch) to ever be
> solved? More importantly, is there an issue in the
first place?
>
> For all use cases where it might make sense to use
multiple indexes, the
> common (as in community and commiters) wisdom is that
one index is enough

I think that's the case mainly because it reflected the
capabilities
of solr at the time... so if someone asks how they create
more than
one index, the natural reaction has been to ask "do you
really need
to?" in the spirit of helping them quickly solve their
problem with
what was available.

Of course, now there is your very ambitious SOLR-215, which
I
personally haven't had the time to look at, but I commend
your great
writeup on it - it really helps to have that type of
detailed overview
when attempting to evaluate such a large patch.

-Yonik

This  http://www.nabble.com/multiple-indices-tf3982573.html
thread  triggers
the question again.
Solr-215 makes it easier to deploy multiple indexes than
using multiple web
applications; but is "easier" enough for not being
just a superfluous
feature?


Henrib wrote:
> 
> 
> 
> The idea of the multiple core/indexes feature has been
discussed in many
> threads and it seems/seemed it has/had some functional
value; how do we
> ensure this value is generic enough for the issue
(& patch) to ever be
> solved? More importantly, is there an issue in the
first place?
> 
> For all use cases where it might make sense to use
multiple indexes, the
> common (as in community and commiters) wisdom is that
one index is enough
> and its schema will accommodate any functional need;
from storing
> documents that have no field in common to documents in
different languages
> or documents with varying publication workflows or
lifetime, etc..., one
> index (schema/config/core) - fits the functional bill.
In very rare hard
> cases, you might have to deploy multiple web
applications.
> 
> If having only one index is never a problem, the
project has no need for a
> patch that introduces multiple indexes. Thus, if the
solr-215 issue is a
> non-issue, what is the process to close it and mark it
as 'invalid'?
> It would be of great community value to state that
thinking about using
> multiple indexes is a mistake within the Solr project
scope; use Lucene or
> else if this is what you need.
> 
> Is/should the single index best practice the sole one
the Solr community
> needs ?
> I should have asked this earlier, still learning...
> 
> 

-- 
View this message in context: http:/
/www.nabble.com/Multiple-indexes-cores-%28aka-solr-215%29-fu
nctional-value--tf3927076.html#a11325717
Sent from the Solr - Dev mailing list archive at
Nabble.com.

On 6/27/07, Henrib <hbiestrogmail.com> wrote:
> This  http://www.nabble.com/multiple-indices-tf3982573.html
thread  triggers
> the question again.
> Solr-215 makes it easier to deploy multiple indexes
than using multiple web
> applications; but is "easier" enough for not
being just a superfluous
> feature?

With a fixed handful of indicies, IMO, no.
Though if one needs to programmatically add new
indicies/schemas,
SOLR-215 becomes interesting.  I don't know how common of a
case that
is though.  There are probably other use cases I've not
considered.

SOLR-215 does seem unrelated to distributed search though.

-Yonik

The starting point for Solr-215 was this 
http://www.nabble.com/Embed
ding-Solr-vs-Lucene%2C-multiple-Solr-cores--tf3572324.html
thread  in April.
I'll still say that some of our customers IT are making
deployment of
multiple webapps something more complex than it should...but
they do!  IT is
in fact managing an internal hosting service and do not want
to deal with
the functional part of the applications. In these
environments, a webapp
that starts a thread & uses the file system is already a
hard sell...

In the same thread, I got the impression that removing the
singletons would
be useful anyway:

hossman_lucene wrote:
> 
>>That said: If anyone is interested in tackling a
patch to eliminate all of
>>the static Singletons i (and many others i suspect)
would be
>>extremely gratefull .. both for how much it would
improve hte reusability
>>of Solr in embedded situatiosn like this, but also
for how it would
>>(hopefully) make hte code eaier to follow for future
developers.
> 

More to your point, programmatic creation of indexes comes
handy if you want
to extend Solr with collection management (rather than
dealing with them
"externally" through scripts).
You can easily add collections and/or languages and manage
the collections
definition/refresh/reindexing within the application; it
seemed like a
natural extension to the project. The main rationale being
that since the
Solr document/field format is most likely not the authoring
format of the
original "resource" (the authored source file),
there is value in declaring
how to transform a "resource" into an Solr-XML
document from its
collection/lang/suffix/etc. But this is beyond Solr-215.
The same could be said for distributed search (aka solr-255)
versus a 'Solr
cluster management' application...

At this point, Solr-215 just looks like a solution to my
problem...
My apologies for the clutter & waste of community
resources.

Yonik Seeley wrote:
> 
> On 6/27/07, Henrib <hbiestrogmail.com> wrote:
>> This  http://www.nabble.com/multiple-indices-tf3982573.html
thread 
>> triggers
>> the question again.
>> Solr-215 makes it easier to deploy multiple indexes
than using multiple
>> web
>> applications; but is "easier" enough for
not being just a superfluous
>> feature?
> 
> With a fixed handful of indicies, IMO, no.
> Though if one needs to programmatically add new
indicies/schemas,
> SOLR-215 becomes interesting.  I don't know how common
of a case that
> is though.  There are probably other use cases I've not
considered.
> 
> SOLR-215 does seem unrelated to distributed search
though.
> 
> -Yonik
> 
> 

-- 
View this message in context: http:/
/www.nabble.com/Multiple-indexes-cores-%28aka-solr-215%29-fu
nctional-value--tf3927076.html#a11327863
Sent from the Solr - Dev mailing list archive at
Nabble.com.

On 6/27/07, Henrib <hbiestrogmail.com> wrote:
> At this point, Solr-215 just looks like a solution to
my problem...
> My apologies for the clutter & waste of community
resources.

Not at all... I don't mean to discourage you. I'm still on
the fence,
and it's a large patch to swallow and figure out all the
implications
of.

Of course, the nature of a volunteer community means that a
good patch
can still sit unused for quite until it happens to intersect
with the
priorities of a committer.

-Yonik

: At this point, Solr-215 just looks like a solution to my
problem...
: My apologies for the clutter & waste of community
resources.

Please, please, PLEASE don't feel that way ... a
functioning
patch and/or a productive discussion is *NEVER* a waste of
resources.
I think Doug Cutting is the first person that told me the
mantra all open
source projects should live by "a patch in the bug
queue that never gets
commited is still a useful patch because it's available to
people that
want it." ... but commiting a patch (especially a big
one) requires taking
on a lot of responsibility to stand behind the
APIs/functionality it
includes and commit to a "best effort" to support
them in a backwards
compatible way for as long as feasible.


On a personal note: Please keep in mind that just because i
make
suggestions to people on how to achieve multiple indexes /
schemas /
doctypes that doesn't require your patch doesn't mean i
think there's
anything wrong with your patch ... i'm just trying to make
sure people are
aware of what's posisble "out of the box".    I
think the appraoch you've
advoated in teh jira issue description makes a lot of sense,
and i kind of
wish Solr had been implemented that way in the begining --
it just wasn't
anticipated.  the real question in my mind is: can we get
from here to
there cleanly, and is your patch the way to do it?

i don't know the answer to that question yet.  but please
don't be
discouraged.



-Hoss