Separate config and index per webapp

On 5/15/06, Michael Levy <LuceneListgmail.com> wrote:
> I'd like to use Solr for a number of separate projects
running on a
> single Tomcat instance; I thought I would have Solr
running a separate
> webapp for each project.
>
> I understand that the default location for the
solrconfig.xml and
> schema.xml files and for the index are under Tomcat's
> current-working-directory/solr/conf
>
> I am hoping I can change the default location for each
webapp.  Thanks!

It's not yet possible, but see this thread:
http://www.mail-
archive.com/solr-devlucene.apache.org/msg00298.html

It looks like JNDI offers the easiest portable way.
The main SolrServlet would then need to be modified to do a
JNDI
lookup and base config off of that.

-Yonik

>On 5/15/06, Michael Levy <LuceneListgmail.com> wrote:
>>I'd like to use Solr for a number of separate
projects running on a
>>single Tomcat instance; I thought I would have Solr
running a separate
>>webapp for each project.
>>
>>I understand that the default location for the
solrconfig.xml and
>>schema.xml files and for the index are under
Tomcat's
>>current-working-directory/solr/conf
>>
>>I am hoping I can change the default location for
each webapp.  Thanks!
>
>It's not yet possible, but see this thread:
>http://www.mail-
archive.com/solr-devlucene.apache.org/msg00298.html
>
>It looks like JNDI offers the easiest portable way.
>The main SolrServlet would then need to be modified to
do a JNDI
>lookup and base config off of that.

I worked around it this way:

http://www.mail-
archive.com/solr-devlucene.apache.org/msg00321.html

-- Ken
-- 
Ken Krugler
Krugle, Inc.
+1 530-210-6378
"Find Code, Find Answers"

On May 15, 2006, at 3:26 PM, Yonik Seeley wrote:

> On 5/15/06, Michael Levy <LuceneListgmail.com> wrote:
>> I'd like to use Solr for a number of separate
projects running on a
>> single Tomcat instance; I thought I would have Solr
running a  
>> separate
>> webapp for each project.
>>
>> I understand that the default location for the
solrconfig.xml and
>> schema.xml files and for the index are under
Tomcat's
>> current-working-directory/solr/conf
>>
>> I am hoping I can change the default location for
each webapp.   
>> Thanks!
>
> It's not yet possible, but see this thread:
> http://www.mail-
archive.com/solr-devlucene.apache.org/msg00298.html
>
> It looks like JNDI offers the easiest portable way.
> The main SolrServlet would then need to be modified to
do a JNDI
> lookup and base config off of that.


The way I did it is I unpacked the WAR and put my config
files into  
WEB-INF/classes/. Then I individually edit each app's
solrconfig.xml  
and specify a unique path for the index directory. This may
not be  
for everyone, but since I had to unpack the WAR anyway...

-MB

Yonik Seeley wrote:
>> I am hoping I can change the default location for
each webapp.  Thanks!
> It's not yet possible, but see this thread:
> http://www.mail-
archive.com/solr-devlucene.apache.org/msg00298.html
If I see it right, if I just rename the webapp to, say,
"solrfoo" then 
it still uses the system property solr.solr.home to search
for the 
configuration, *not* solrfoo.solr.home, right?
I'm searching for a way to have multiple webapps with
different 
configuration, too. I would really appreciate if that could
be made 
possible. (And sorry, I'd really like to do it myself, but
my java 
knowledge doesn not suffice for that.)

Another thing I would like to see is a complete detachment
of the solr 
configuration of that of the servlet container. Currently I
have to 
change the path to the configuration files by setting
solr.solr.home or 
(even worse!) by starting Tomcat (which I use) from it's
base home dir.
A while ago I proposed to put solr's config into /etc/solr
(for linux). 
It was easily done (even for me) to add this directory to
the places 
being searched in Config.java. I thing if this is put in
*additionally* 
it should be no problem even for those people who just want
to try out 
solr and have no root privileges.

Marcus

: If I see it right, if I just rename the webapp to, say,
"solrfoo" then
: it still uses the system property solr.solr.home to search
for the
: configuration, *not* solrfoo.solr.home, right?

correct .. we thought we can impliment something that looked
at the war
file name easily ... but then we were set straight -- there
is no portable
way to do that, hence we came up with the current JNDI plan
which isn't
quite as "out of the box" as we had hoped, but
it has the advantage of
being possible.

: Another thing I would like to see is a complete detachment
of the solr
: configuration of that of the servlet container. Currently
I have to

I don't know that we'll ever be able to make configuring
Solr completely
detatched from configuring the servlet container -- other
then the
simplest method of putting your.  Personally i don't think
that should be
a major goal: a well tuned Solr installation is going to
require that you
consider/configure your servlet container's heap size to
meet your needs
anyway.

: change the path to the configuration files by setting
solr.solr.home or
: (even worse!) by starting Tomcat (which I use) from it's
base home dir.

just to clarify: if you only want one instance of Solr on
the port, you
do't *have* to start tomcat from it's base directory ...
you just have to
make sure the "solr" directory is in whatever
the current working
directory is when you do start it.  many servlet container
start scripts
already make sure to set the working directory to the
installation
directory before starting up the server, so maybe that's
the source of the
confusion?

: A while ago I proposed to put solr's config into
/etc/solr (for linux).
: It was easily done (even for me) to add this directory to
the places
: being searched in Config.java. I thing if this is put in
*additionally*
: it should be no problem even for those people who just
want to try out
: solr and have no root privileges.

1) I'm not sure how this would work cleanly for people on
Windows boxes
2) this still wouldn't make it possible to suport multiple
webapps on a
single port (it would acctually make it hard to support
multiple ports on
the same box).

If the JNDI approach gets implimented, then it should make
it easy for you
to specify /etc/solr (or any other directory) as your config
directory
with a one line change to your tomcat configuration.

But please don't let me discourage you from experiencing
with this
approach yourself, and submitting a patch that does what
you'd like ...
Perhaps I'm just missunderstanding.




-Hoss

I copied solrconfig.xml, schema.xml, synonyms.txt,
protwords.txt and 
stopwords.txt into the WEB-INF/classes directory and the
webapp is 
reading those files.

I was thinking that if I uncommented the indexDir element in

solrconfig.xml, it would then use that directory in which to
place the index
The commented-out default value is:
  <indexDir>index</indexDir>

I tried changing the value to:
  <indexDir>indexXXX</indexDir>

However this is being ignored; the index directory is still
being 
created under Tomcat's
current-working-directory/solr/conf/index   (In 
my case I'm testing under Windows and it's C:\Tomcat
5.5\solr\data\index)

Any ideas?  Thanks.

Mike Baranczak wrote:
> On May 15, 2006, at 3:26 PM, Yonik Seeley wrote:
>
>> On 5/15/06, Michael Levy <LuceneListgmail.com> wrote:
>>> I'd like to use Solr for a number of separate
projects running on a
>>> single Tomcat instance; I thought I would have
Solr running a separate
>>> webapp for each project.
>>>
>>> I understand that the default location for the
solrconfig.xml and
>>> schema.xml files and for the index are under
Tomcat's
>>> current-working-directory/solr/conf
>>>
>>> I am hoping I can change the default location
for each webapp.  Thanks!
>>
>> It's not yet possible, but see this thread:
>> http://www.mail-
archive.com/solr-devlucene.apache.org/msg00298.html
>>
>> It looks like JNDI offers the easiest portable way.
>> The main SolrServlet would then need to be modified
to do a JNDI
>> lookup and base config off of that.
>
>
> The way I did it is I unpacked the WAR and put my
config files into 
> WEB-INF/classes/. Then I individually edit each app's
solrconfig.xml 
> and specify a unique path for the index directory. This
may not be for 
> everyone, but since I had to unpack the WAR anyway...
>
> -MB
>
>

Try using the latest nightly build, and changing indexDir to
dataDir:

  <!-- Used to specify an alternate directory to hold all
index data
       other than the default ./solr/data
       If replication is in use, this should match the
replication configuration
. -->
  <!--
  <dataDir>./solr/data</dataDir>
  -->

-Yonik

On 5/17/06, Michael Levy <LuceneListgmail.com> wrote:
> I copied solrconfig.xml, schema.xml, synonyms.txt,
protwords.txt and
> stopwords.txt into the WEB-INF/classes directory and
the webapp is
> reading those files.
>
> I was thinking that if I uncommented the indexDir
element in
> solrconfig.xml, it would then use that directory in
which to place the index
> The commented-out default value is:
>   <indexDir>index</indexDir>
>
> I tried changing the value to:
>   <indexDir>indexXXX</indexDir>
>
> However this is being ignored; the index directory is
still being
> created under Tomcat's
current-working-directory/solr/conf/index   (In
> my case I'm testing under Windows and it's C:\Tomcat
5.5\solr\data\index)
>
> Any ideas?  Thanks.
>
> Mike Baranczak wrote:
> > On May 15, 2006, at 3:26 PM, Yonik Seeley wrote:
> >
> >> On 5/15/06, Michael Levy <LuceneListgmail.com> wrote:
> >>> I'd like to use Solr for a number of
separate projects running on a
> >>> single Tomcat instance; I thought I would
have Solr running a separate
> >>> webapp for each project.
> >>>
> >>> I understand that the default location for
the solrconfig.xml and
> >>> schema.xml files and for the index are
under Tomcat's
> >>> current-working-directory/solr/conf
> >>>
> >>> I am hoping I can change the default
location for each webapp.  Thanks!
> >>
> >> It's not yet possible, but see this thread:
> >> http://www.mail-
archive.com/solr-devlucene.apache.org/msg00298.html
> >>
> >> It looks like JNDI offers the easiest portable
way.
> >> The main SolrServlet would then need to be
modified to do a JNDI
> >> lookup and base config off of that.
> >
> >
> > The way I did it is I unpacked the WAR and put my
config files into
> > WEB-INF/classes/. Then I individually edit each
app's solrconfig.xml
> > and specify a unique path for the index directory.
This may not be for
> > everyone, but since I had to unpack the WAR
anyway...
> >
> > -MB

Changing to <dataDir> works fine with my version from
2006-05-15

Thanks for your help, this is just what I needed.

Yonik Seeley wrote:
> Try using the latest nightly build, and changing
indexDir to dataDir:
>
>  <!-- Used to specify an alternate directory to hold
all index data
>       other than the default ./solr/data
>       If replication is in use, this should match the
replication 
> configuration
> . -->
>  <!--
>  <dataDir>./solr/data</dataDir>
>  -->
>
> -Yonik
>
> On 5/17/06, Michael Levy <LuceneListgmail.com> wrote:
>> I copied solrconfig.xml, schema.xml, synonyms.txt,
protwords.txt and
>> stopwords.txt into the WEB-INF/classes directory
and the webapp is
>> reading those files.
>>
>> I was thinking that if I uncommented the indexDir
element in
>> solrconfig.xml, it would then use that directory in
which to place 
>> the index
>> The commented-out default value is:
>>   <indexDir>index</indexDir>
>>
>> I tried changing the value to:
>>   <indexDir>indexXXX</indexDir>
>>
>> However this is being ignored; the index directory
is still being
>> created under Tomcat's
current-working-directory/solr/conf/index   (In
>> my case I'm testing under Windows and it's
C:\Tomcat 
>> 5.5\solr\data\index)
>>
>> Any ideas?  Thanks.
>>
>> Mike Baranczak wrote:
>> > On May 15, 2006, at 3:26 PM, Yonik Seeley
wrote:
>> >
>> >> On 5/15/06, Michael Levy
<LuceneListgmail.com> wrote:
>> >>> I'd like to use Solr for a number of
separate projects running on a
>> >>> single Tomcat instance; I thought I
would have Solr running a 
>> separate
>> >>> webapp for each project.
>> >>>
>> >>> I understand that the default location
for the solrconfig.xml and
>> >>> schema.xml files and for the index are
under Tomcat's
>> >>> current-working-directory/solr/conf
>> >>>
>> >>> I am hoping I can change the default
location for each webapp.  
>> Thanks!
>> >>
>> >> It's not yet possible, but see this
thread:
>> >> http://www.mail-
archive.com/solr-devlucene.apache.org/msg00298.html
>> >>
>> >> It looks like JNDI offers the easiest
portable way.
>> >> The main SolrServlet would then need to be
modified to do a JNDI
>> >> lookup and base config off of that.
>> >
>> >
>> > The way I did it is I unpacked the WAR and put
my config files into
>> > WEB-INF/classes/. Then I individually edit
each app's solrconfig.xml
>> > and specify a unique path for the index
directory. This may not be for
>> > everyone, but since I had to unpack the WAR
anyway...
>> >
>> > -MB
>

Yeah, the config that I have (based on the repo version from
a couple  
weeks ago) has <dataDir> set, and it works as
expected. I don't know  
where the indexDir came from.


On May 17, 2006, at 2:56 PM, Yonik Seeley wrote:

> Try using the latest nightly build, and changing
indexDir to dataDir:
>
>  <!-- Used to specify an alternate directory to hold
all index data
>       other than the default ./solr/data
>       If replication is in use, this should match the
replication  
> configuration
> . -->
>  <!--
>  <dataDir>./solr/data</dataDir>
>  -->
>
> -Yonik
>
> On 5/17/06, Michael Levy <LuceneListgmail.com> wrote:
>> I copied solrconfig.xml, schema.xml, synonyms.txt,
protwords.txt and
>> stopwords.txt into the WEB-INF/classes directory
and the webapp is
>> reading those files.
>>
>> I was thinking that if I uncommented the indexDir
element in
>> solrconfig.xml, it would then use that directory in
which to place  
>> the index
>> The commented-out default value is:
>>   <indexDir>index</indexDir>
>>
>> I tried changing the value to:
>>   <indexDir>indexXXX</indexDir>
>>
>> However this is being ignored; the index directory
is still being
>> created under Tomcat's
current-working-directory/solr/conf/index    
>> (In
>> my case I'm testing under Windows and it's
C:\Tomcat 5.5\solr\data 
>> \index)
>>
>> Any ideas?  Thanks.
>>
>> Mike Baranczak wrote:
>> > On May 15, 2006, at 3:26 PM, Yonik Seeley
wrote:
>> >
>> >> On 5/15/06, Michael Levy
<LuceneListgmail.com> wrote:
>> >>> I'd like to use Solr for a number of
separate projects running  
>> on a
>> >>> single Tomcat instance; I thought I
would have Solr running a  
>> separate
>> >>> webapp for each project.
>> >>>
>> >>> I understand that the default location
for the solrconfig.xml and
>> >>> schema.xml files and for the index are
under Tomcat's
>> >>> current-working-directory/solr/conf
>> >>>
>> >>> I am hoping I can change the default
location for each  
>> webapp.  Thanks!
>> >>
>> >> It's not yet possible, but see this
thread:
>> >> http://www.mail-
archive.com/solr-devlucene.apache.org/ 
>> msg00298.html
>> >>
>> >> It looks like JNDI offers the easiest
portable way.
>> >> The main SolrServlet would then need to be
modified to do a JNDI
>> >> lookup and base config off of that.
>> >
>> >
>> > The way I did it is I unpacked the WAR and put
my config files into
>> > WEB-INF/classes/. Then I individually edit
each app's  
>> solrconfig.xml
>> > and specify a unique path for the index
directory. This may not  
>> be for
>> > everyone, but since I had to unpack the WAR
anyway...
>> >
>> > -MB

Chris Hostetter wrote:
> correct .. we thought we can impliment something that
looked at the war
> file name easily ... but then we were set straight --
there is no portable
> way to do that, hence we came up with the current JNDI
plan which isn't
> quite as "out of the box" as we had hoped,
but it has the advantage of
> being possible.
Yes, I observed the discussion on the developer mailing list
for a while 
and was suprised to read that there isn't an easy solution
for this problem.

> I don't know that we'll ever be able to make
configuring Solr completely
> detatched from configuring the servlet container --
other then the
> simplest method of putting your.  Personally i don't
think that should be
> a major goal: a well tuned Solr installation is going
to require that you
> consider/configure your servlet container's heap size
to meet your needs
> anyway.
Good point. Currently I'm using the solr.solr.home system
property and 
besides the heap size it is the only Solr specific
configuration I have 
to do with Tomcat. So I can live with that.

> just to clarify: if you only want one instance of Solr
on the port, you
> do't *have* to start tomcat from it's base directory
I know, I just wanted to point out that somehow Tomcat is
involved in 
the Solr configuration.

> ... you just have to
> make sure the "solr" directory is in
whatever the current working
> directory is when you do start it.
But what if another webapp needs the server to be started
from 
/some/directory/it/likes ?

> If the JNDI approach gets implimented, then it should
make it easy for you
> to specify /etc/solr (or any other directory) as your
config directory
> with a one line change to your tomcat configuration.
I'm looking forward to that. 

Thanks,
Marcus