Make/install as non-root

Good afternoon!

First, as this is my first actual post (after a lot of
lurking), a brief
introduction: I'm the (relatively) new senior system
administrator at
Gossamer Threads, working for Alex Krohn.

We run Bricolage installs in a non-root environment, and
have a series
of patches and hacks to make that work. I've been cleaning
it up a lot
over the last few weeks and trying to make patches to send
back
upstream. The basic idea is that, short of creating the
database and the
database user, the entire install should be able to be run
in userspace.
I had these working against 1.10.4, and am now rewriting
bits of them
against the current svn checkout.

This mostly requires minor changes; for example, getting
bin/Makefile.PL
and lib/Makefile.PL to pass BRICOLAGE_ROOT as PREFIX, if
that variable
is defined in config.db.

For most database-related tasks, I've had it work as the
bricolage user
rather than as root (unless $> == 0).

If this seems useful, I'll send along patches and a full
list of changes
once I finish adapting it to the current svn.

-Brian

On Apr 18, 2008, at 17:27, Brian Smith wrote:

> First, as this is my first actual post (after a lot of
lurking), a  
> brief
> introduction: I'm the (relatively) new senior system
administrator at
> Gossamer Threads, working for Alex Krohn.
>
> We run Bricolage installs in a non-root environment,
and have a series
> of patches and hacks to make that work. I've been
cleaning it up a lot
> over the last few weeks and trying to make patches to
send back
> upstream. The basic idea is that, short of creating the
database and  
> the
> database user, the entire install should be able to be
run in  
> userspace.
> I had these working against 1.10.4, and am now
rewriting bits of them
> against the current svn checkout.

Trunk? Awesome.

> This mostly requires minor changes; for example,
getting bin/ 
> Makefile.PL
> and lib/Makefile.PL to pass BRICOLAGE_ROOT as PREFIX,
if that variable
> is defined in config.db.

Ah, that's smart.

> For most database-related tasks, I've had it work as
the bricolage  
> user
> rather than as root (unless $> == 0).

Yeah, that's where it will get tricky. Stupid ident auth.

> If this seems useful, I'll send along patches and a
full list of  
> changes
> once I finish adapting it to the current svn.

I look forward to seeing them. And welcome!

Best,

David

Here's a few to get started with...

The only outstanding issue is that mysql.sql has a SET
GLOBAL in line 31
after make dist and some CREATE TRIGGERs; that won't work as
user, but
everything else does. If I temporarily grant SUPER, it'll
work.

All the Postgres stuff works in userspace after patching.
All patches
are from trunk as of an hour ago.

patch1.diff - just adds /usr/lib/httpd to the initial Apache
module
check, which is where Apache modules reside on our servers
(and I think
on Gentoo in general).

patch2.diff - adds the BRICOLAGE_ROOT / PREFIX tie-in that I
mentioned
in my previous e-mail.

patch3.diff - does all the database modifications necessary
for make
install to work in userspace:
 * adds an extra question in the DB prober to ask if the
installer
should create the database and database user;
 * modifies the DB loaders so that they only load the data,
not create
the databases, if create_db is 0; and
 * skips the DB grant process entirely if create_db is 0

(This is the ugliest piece of what I've done here, and I'm
still not
convinced it's the best way to do it. It works, but there
may be better
ways to go.)

patch4.diff - makes 'make uninstall' work by warning the
user, if
they're not root, that it won't be able to drop the database
(and thus
just uninstalls the files).

With this -- aside from that SUPER thing -- make dist, make,
make
install, and make uninstall all work. I still need to play
with make
upgrade and make clone, which I plan to tackle this weekend
or early
next week.

-Brian

On Fri, 2008-04-18 at 17:33 -0700, David E. Wheeler wrote:
> On Apr 18, 2008, at 17:27, Brian Smith wrote:
> 
> > This mostly requires minor changes; for example,
getting bin/ 
> > Makefile.PL
> > and lib/Makefile.PL to pass BRICOLAGE_ROOT as
PREFIX, if that variable
> > is defined in config.db.
> 
> Ah, that's smart.
> 
> > For most database-related tasks, I've had it work
as the bricolage  
> > user
> > rather than as root (unless $> == 0).
> 
> Yeah, that's where it will get tricky. Stupid ident
auth.
> 
> > If this seems useful, I'll send along patches and
a full list of  
> > changes
> > once I finish adapting it to the current svn.
> 
> I look forward to seeing them. And welcome!
> 
> Best,
> 
> David
> 

*sigh* I'll try that again, actually attaching the files
this time . . .

-Brian

On Fri, 2008-04-18 at 19:26 -0700, Brian Smith wrote:
> Here's a few to get started with...
> 
> The only outstanding issue is that mysql.sql has a SET
GLOBAL in line 31
> after make dist and some CREATE TRIGGERs; that won't
work as user, but
> everything else does. If I temporarily grant SUPER,
it'll work.
> 
> All the Postgres stuff works in userspace after
patching. All patches
> are from trunk as of an hour ago.
> 
> patch1.diff - just adds /usr/lib/httpd to the initial
Apache module
> check, which is where Apache modules reside on our
servers (and I think
> on Gentoo in general).
> 
> patch2.diff - adds the BRICOLAGE_ROOT / PREFIX tie-in
that I mentioned
> in my previous e-mail.
> 
> patch3.diff - does all the database modifications
necessary for make
> install to work in userspace:
>  * adds an extra question in the DB prober to ask if
the installer
> should create the database and database user;
>  * modifies the DB loaders so that they only load the
data, not create
> the databases, if create_db is 0; and
>  * skips the DB grant process entirely if create_db is
0
> 
> (This is the ugliest piece of what I've done here, and
I'm still not
> convinced it's the best way to do it. It works, but
there may be better
> ways to go.)
> 
> patch4.diff - makes 'make uninstall' work by warning
the user, if
> they're not root, that it won't be able to drop the
database (and thus
> just uninstalls the files).
> 
> With this -- aside from that SUPER thing -- make dist,
make, make
> install, and make uninstall all work. I still need to
play with make
> upgrade and make clone, which I plan to tackle this
weekend or early
> next week.
> 
> -Brian
> 

On Apr 18, 2008, at 19:26, Brian Smith wrote:

> Here's a few to get started with...
>
> The only outstanding issue is that mysql.sql has a SET
GLOBAL in  
> line 31
> after make dist and some CREATE TRIGGERs; that won't
work as user, but
> everything else does. If I temporarily grant SUPER,
it'll work.

So is that possible? We do need to have those settings in
place for  
all use of Bricolage. If they can just be limited to the
Bricolage  
database itself, rather than global, then that's fine.

> All the Postgres stuff works in userspace after
patching. All patches
> are from trunk as of an hour ago.
>
> patch1.diff - just adds /usr/lib/httpd to the initial
Apache module
> check, which is where Apache modules reside on our
servers (and I  
> think
> on Gentoo in general).

Crimony, I can't tell you how I hate that every fucking
distribution  
keeps shit in different places.

> patch2.diff - adds the BRICOLAGE_ROOT / PREFIX tie-in
that I mentioned
> in my previous e-mail.
>
> patch3.diff - does all the database modifications
necessary for make
> install to work in userspace:
> * adds an extra question in the DB prober to ask if the
installer
> should create the database and database user;
> * modifies the DB loaders so that they only load the
data, not create
> the databases, if create_db is 0; and
> * skips the DB grant process entirely if create_db is
0

Thanks. I'll take a look at these tomorrow, most likely.

> (This is the ugliest piece of what I've done here, and
I'm still not
> convinced it's the best way to do it. It works, but
there may be  
> better
> ways to go.)

I'll take a look. The whole installer is ugly, though, so I
can't say  
I'm surprised.

> patch4.diff - makes 'make uninstall' work by warning
the user, if
> they're not root, that it won't be able to drop the
database (and thus
> just uninstalls the files).

Well, can't the user drop the database if they don't have
system root,  
but do have database root access? Is the latter what you
meant?

> With this -- aside from that SUPER thing -- make dist,
make, make
> install, and make uninstall all work. I still need to
play with make
> upgrade and make clone, which I plan to tackle this
weekend or early
> next week.

Awesome, thanks! "SUPER thing"?

Best,

David

On Sun, Apr 20, 2008 at 10:29:17AM -0700, David E. Wheeler
wrote:
> On Apr 18, 2008, at 19:26, Brian Smith wrote:
> 
> >Here's a few to get started with...
> >
> >The only outstanding issue is that mysql.sql has a
SET GLOBAL in  
> >line 31
> >after make dist and some CREATE TRIGGERs; that
won't work as user, but
> >everything else does. If I temporarily grant SUPER,
it'll work.
> 
> So is that possible? We do need to have those settings
in place for  
> all use of Bricolage. If they can just be limited to
the Bricolage  
> database itself, rather than global, then that's fine.
> 
> >All the Postgres stuff works in userspace after
patching. All patches
> >are from trunk as of an hour ago.
> >
> >patch1.diff - just adds /usr/lib/httpd to the
initial Apache module
> >check, which is where Apache modules reside on our
servers (and I  
> >think
> >on Gentoo in general).
> 
> Crimony, I can't tell you how I hate that every fucking
distribution  
> keeps shit in different places.

If this is overrideable via an ENV variable, that'd be
excellent.

> >patch2.diff - adds the BRICOLAGE_ROOT / PREFIX
tie-in that I mentioned
> >in my previous e-mail.
> >
> >patch3.diff - does all the database modifications
necessary for make
> >install to work in userspace:
> >* adds an extra question in the DB prober to ask if
the installer
> >should create the database and database user;
> >* modifies the DB loaders so that they only load
the data, not create
> >the databases, if create_db is 0; and
> >* skips the DB grant process entirely if create_db
is 0
> 
> Thanks. I'll take a look at these tomorrow, most
likely.
> 
> >(This is the ugliest piece of what I've done here,
and I'm still not
> >convinced it's the best way to do it. It works, but
there may be  
> >better
> >ways to go.)
> 
> I'll take a look. The whole installer is ugly, though,
so I can't say  
> I'm surprised.
> 
> >patch4.diff - makes 'make uninstall' work by
warning the user, if
> >they're not root, that it won't be able to drop the
database (and thus
> >just uninstalls the files).
> 
> Well, can't the user drop the database if they don't
have system root,  
> but do have database root access? Is the latter what
you meant?
> 
> >With this -- aside from that SUPER thing -- make
dist, make, make
> >install, and make uninstall all work. I still need
to play with make
> >upgrade and make clone, which I plan to tackle this
weekend or early
> >next week.
> 
> Awesome, thanks! "SUPER thing"?
> 
> Best,
> 
> David
> 

-- 

Brad Harder,
Method Digital Logic
http://www.methodlogic.net

On Sun, 2008-04-20 at 10:48 -0700, bharder wrote:
> > >patch1.diff - just adds /usr/lib/httpd to the
initial Apache module
> > >check, which is where Apache modules reside on
our servers (and I  
> > >think
> > >on Gentoo in general).
> > 
> > Crimony, I can't tell you how I hate that every
fucking distribution  
> > keeps shit in different places.
> 
> If this is overrideable via an ENV variable, that'd be
excellent.

Alternately, how about having the installer ask? I can throw
a patch for
that today if that sounds good.

(I'm thinking -- it does its detection as best it can, but
if it fails,
instead of just failing outright, it asks where the Apache
module
directory is. Then I could type '/usr/lib/httpd' and off we
go.)

-Brian

On Apr 21, 2008, at 11:12, Brian Smith wrote:

>> If this is overrideable via an ENV variable, that'd
be excellent.
>
> Alternately, how about having the installer ask? I can
throw a patch  
> for
> that today if that sounds good.

AFAIK, it *does* ask.

> (I'm thinking -- it does its detection as best it can,
but if it  
> fails,
> instead of just failing outright, it asks where the
Apache module
> directory is. Then I could type '/usr/lib/httpd' and
off we go.)

It does ask.

Best,

David

It asks the path to the Apache binary--which here
is /usr/sbin/httpd--but it never asks the path to the
Apache
libraries/modules. So I give it the correct binary path, and
then I see:

---
==> Probing Apache Configuration <==

Extracting configuration data from `/usr/sbin/httpd -V`.
Extracting static module list from `/usr/sbin/httpd -l`.
Your Apache supports loadable modules (DSOs).
Checking for required Apache modules...
############################################################
###################

The following Apache modules are required by Bricolage and
are missing from your installation:
        mod_perl
        mod_log_config
        mod_mime
        mod_alias


############################################################
###################
make: *** [apache.db] Error 1
briandev32 bricolage-1.11.0 $ 
---

This is the case in both 1.10.4 and in the current svn
trunk.

-Brian

On Mon, 2008-04-21 at 11:17 -0700, David E. Wheeler wrote:
> On Apr 21, 2008, at 11:12, Brian Smith wrote:
> 
> >> If this is overrideable via an ENV variable,
that'd be excellent.
> >
> > Alternately, how about having the installer ask? I
can throw a patch  
> > for
> > that today if that sounds good.
> 
> AFAIK, it *does* ask.
> 
> > (I'm thinking -- it does its detection as best it
can, but if it  
> > fails,
> > instead of just failing outright, it asks where
the Apache module
> > directory is. Then I could type '/usr/lib/httpd'
and off we go.)
> 
> It does ask.
> 
> Best,
> 
> David
> 

On Mon, Apr 21, 2008 at 11:12:58AM -0700, Brian Smith
wrote:
> 
> On Sun, 2008-04-20 at 10:48 -0700, bharder wrote:
> > > >patch1.diff - just adds /usr/lib/httpd to
the initial Apache module
> > > >check, which is where Apache modules
reside on our servers (and I  
> > > >think
> > > >on Gentoo in general).
> > > 
> > > Crimony, I can't tell you how I hate that
every fucking distribution  
> > > keeps shit in different places.
> > 
> > If this is overrideable via an ENV variable,
that'd be excellent.
> 
> Alternately, how about having the installer ask? I can
throw a patch for
> that today if that sounds good.

I think a way to do a non-interactive configure would be
nice. Setting an ENV variable fits the bill in this case.
That way, for make(1)-based installs (ie: FreeBSD, NetBSD,
OpenBSD, Gentoo, etc.), one can have an variable set for
that arch, and everything will Just Work. An interactive
alternative in this case is useless. If there's no ENV var
(or similar -- environment vars are just the first thing I
thought of, and seem both capable and simple enough), then
package managers will need to start patching the installer.

> (I'm thinking -- it does its detection as best it can,
but if it fails,
> instead of just failing outright, it asks where the
Apache module
> directory is. Then I could type '/usr/lib/httpd' and
off we go.)
> 
> -Brian
> 

-- 

Brad Harder,
Method Digital Logic
http://www.methodlogic.net