Multi-server distributed MediaWiki

It's "easy" to mirror a MediaWiki from one primary
server to a number
of secondary servers, but is it possible to have multiple
primary servers?

Example: 10 servers and users can make changes on ANY of the
10
servers. Every night, the servers rsync to each other as
follows:

1. If server X's version hasn't changed all day and server
Y's version
HAS changed, server X accepts server Y's version.

2. If both server X's and server Y's versions have changed,
automatic
CVS style merging is used to resolve the changes.

3. If CVS style merging yields a conflict, the site
maintainer is
notified and must merge the two files manually (I'm thinking
of a
creating a small site, so this shouldn't be too painful)

I realize the rules above only work for 2 servers -- is
there a clever
version of this for n servers (n>2)?

-- 
We're just a Bunch Of Regular Guys, a collective group
that's trying
to understand and assimilate technology. We feel that
resistance to
new ideas and technology is unwise and ultimately futile.
_______________________________________________
MediaWiki-l mailing list
MediaWiki-lWikimedia.org
http://mail.wikipedia.org/mailman/listinfo/mediawiki-l

On 12/31/06, Kelly Jones <kelly.terry.jonesgmail.com> wrote:
> It's "easy" to mirror a MediaWiki from one
primary server to a number
> of secondary servers, but is it possible to have
multiple primary servers?
>
> Example: 10 servers and users can make changes on ANY
of the 10
> servers. Every night, the servers rsync to each other
as follows:
>
> 1. If server X's version hasn't changed all day and
server Y's version
> HAS changed, server X accepts server Y's version.
>
> 2. If both server X's and server Y's versions have
changed, automatic
> CVS style merging is used to resolve the changes.
>
> 3. If CVS style merging yields a conflict, the site
maintainer is
> notified and must merge the two files manually (I'm
thinking of a
> creating a small site, so this shouldn't be too
painful)
>
> I realize the rules above only work for 2 servers -- is
there a clever
> version of this for n servers (n>2)?

What are you trying to accomplish by doing that?

The way the data is in the databases, it's a little more
hard than that.

How well do you understand clustering theory?


-- 
-george william herbert
george.herbertgmail.com
_______________________________________________
MediaWiki-l mailing list
MediaWiki-lWikimedia.org
http://mail.wikipedia.org/mailman/listinfo/mediawiki-l

Hi,

May be you can try to have a look to mysql-cluster feature
...

But, i guess that you must adapt/change the mediawiki source
code il  
order to make sql requests mysql-cluster compliant ... 

should be hard job ... but funny

Best regards

Arnaud.

Le 2 janv. 07 à 01:31, George Herbert a écrit :

> On 12/31/06, Kelly Jones <kelly.terry.jonesgmail.com> wrote:
>> It's "easy" to mirror a MediaWiki from
one primary server to a number
>> of secondary servers, but is it possible to have
multiple primary  
>> servers?
>>
>> Example: 10 servers and users can make changes on
ANY of the 10
>> servers. Every night, the servers rsync to each
other as follows:
>>
>> 1. If server X's version hasn't changed all day and
server Y's  
>> version
>> HAS changed, server X accepts server Y's version.
>>
>> 2. If both server X's and server Y's versions have
changed, automatic
>> CVS style merging is used to resolve the changes.
>>
>> 3. If CVS style merging yields a conflict, the site
maintainer is
>> notified and must merge the two files manually (I'm
thinking of a
>> creating a small site, so this shouldn't be too
painful)
>>
>> I realize the rules above only work for 2 servers
-- is there a  
>> clever
>> version of this for n servers (n>2)?
>
> What are you trying to accomplish by doing that?
>
> The way the data is in the databases, it's a little
more hard than  
> that.
>
> How well do you understand clustering theory?
>
>
> -- 
> -george william herbert
> george.herbertgmail.com
> _______________________________________________
> MediaWiki-l mailing list
> MediaWiki-lWikimedia.org
> http://mail.wikipedia.org/mailman/listinfo/mediawiki-l
>

_______________________________________________
MediaWiki-l mailing list
MediaWiki-lWikimedia.org
http://mail.wikipedia.org/mailman/listinfo/mediawiki-l

Hello Kelly Jones,

It seems to me that what you are trying to accomplish is
already part
of MediaWiki -- just have all of the ten MediaWikis use the
same
server, or set up each section of articles on a different
interwiki-linked database.  This would remove the need to
rsync
between each of the servers, and it would remove the need
for you to
manually change anything.  The conflicts would not happen,
because
everything would be done in real time.

What are you trying to do?

Kasimir

On 1/1/07, KlinT <klintklintcentral.net> wrote:
> Hi,
>
> May be you can try to have a look to mysql-cluster
feature ...
>
> But, i guess that you must adapt/change the mediawiki
source code il
> order to make sql requests mysql-cluster compliant ...

>
> should be hard job ... but funny
>
> Best regards
>
> Arnaud.
>
> Le 2 janv. 07 à 01:31, George Herbert a écrit :
>
> > On 12/31/06, Kelly Jones <kelly.terry.jonesgmail.com> wrote:
> >> It's "easy" to mirror a MediaWiki
from one primary server to a number
> >> of secondary servers, but is it possible to
have multiple primary
> >> servers?
> >>
> >> Example: 10 servers and users can make changes
on ANY of the 10
> >> servers. Every night, the servers rsync to
each other as follows:
> >>
> >> 1. If server X's version hasn't changed all
day and server Y's
> >> version
> >> HAS changed, server X accepts server Y's
version.
> >>
> >> 2. If both server X's and server Y's versions
have changed, automatic
> >> CVS style merging is used to resolve the
changes.
> >>
> >> 3. If CVS style merging yields a conflict, the
site maintainer is
> >> notified and must merge the two files manually
(I'm thinking of a
> >> creating a small site, so this shouldn't be
too painful)
> >>
> >> I realize the rules above only work for 2
servers -- is there a
> >> clever
> >> version of this for n servers (n>2)?
> >
> > What are you trying to accomplish by doing that?
> >
> > The way the data is in the databases, it's a
little more hard than
> > that.
> >
> > How well do you understand clustering theory?
> >
> >
> > --
> > -george william herbert
> > george.herbertgmail.com
> > _______________________________________________
> > MediaWiki-l mailing list
> > MediaWiki-lWikimedia.org
> > http://mail.wikipedia.org/mailman/listinfo/mediawiki-l
> >
>
> _______________________________________________
> MediaWiki-l mailing list
> MediaWiki-lWikimedia.org
> http://mail.wikipedia.org/mailman/listinfo/mediawiki-l
>


-- 
Kasimir Gabert
_______________________________________________
MediaWiki-l mailing list
MediaWiki-lWikimedia.org
http://mail.wikipedia.org/mailman/listinfo/mediawiki-l

I'm trying to run mediawiki on a free server, but I'm
worried about
bandwidth limits.

I'd like to create mirror sites or let other people create
mirror
sites (the content will be 100% open source) and have a
"master site"
that "randomly" redirects people to a mirror.

The mirror sites should not be "read-only". People
should be able to
edit content on the mirror sites and have their changes
pushed to the
other mirror sites.

The mirror servers should be as independent as possible:
geographically diverse, on different backbones, each using
their own
MySQL server, owned by different people, etc. Some of the
mirror
servers may be free, others may be paid hosting, others may
be
dedicated servers, etc.

The only thing they'd have in common is they'd all be
running
mediawiki and some cron script that merges changed data in
from all
the mirrors.

Thoughts?

-- 
We're just a Bunch Of Regular Guys, a collective group
that's trying
to understand and assimilate technology. We feel that
resistance to
new ideas and technology is unwise and ultimately futile.

On 1/1/07, Kasimir Gabert <kasimir.ggmail.com> wrote:
> Hello Kelly Jones,
>
> It seems to me that what you are trying to accomplish
is already part
> of MediaWiki -- just have all of the ten MediaWikis use
the same
> server, or set up each section of articles on a
different
> interwiki-linked database.  This would remove the need
to rsync
> between each of the servers, and it would remove the
need for you to
> manually change anything.  The conflicts would not
happen, because
> everything would be done in real time.
>
> What are you trying to do?
>
> Kasimir
>
> On 1/1/07, KlinT <klintklintcentral.net> wrote:
> > Hi,
> >
> > May be you can try to have a look to mysql-cluster
feature ...
> >
> > But, i guess that you must adapt/change the
mediawiki source code il
> > order to make sql requests mysql-cluster compliant
... 
> >
> > should be hard job ... but funny
> >
> > Best regards
> >
> > Arnaud.
> >
> > Le 2 janv. 07 à 01:31, George Herbert a écrit :
> >
> > > On 12/31/06, Kelly Jones
<kelly.terry.jonesgmail.com> wrote:
> > >> It's "easy" to mirror a
MediaWiki from one primary server to a number
> > >> of secondary servers, but is it possible
to have multiple primary
> > >> servers?
> > >>
> > >> Example: 10 servers and users can make
changes on ANY of the 10
> > >> servers. Every night, the servers rsync
to each other as follows:
> > >>
> > >> 1. If server X's version hasn't changed
all day and server Y's
> > >> version
> > >> HAS changed, server X accepts server Y's
version.
> > >>
> > >> 2. If both server X's and server Y's
versions have changed, automatic
> > >> CVS style merging is used to resolve the
changes.
> > >>
> > >> 3. If CVS style merging yields a
conflict, the site maintainer is
> > >> notified and must merge the two files
manually (I'm thinking of a
> > >> creating a small site, so this shouldn't
be too painful)
> > >>
> > >> I realize the rules above only work for 2
servers -- is there a
> > >> clever
> > >> version of this for n servers (n>2)?
> > >
> > > What are you trying to accomplish by doing
that?
> > >
> > > The way the data is in the databases, it's a
little more hard than
> > > that.
> > >
> > > How well do you understand clustering theory?
> > >
> > >
> > > --
> > > -george william herbert
> > > george.herbertgmail.com
> > >
_______________________________________________
> > > MediaWiki-l mailing list
> > > MediaWiki-lWikimedia.org
> > > http://mail.wikipedia.org/mailman/listinfo/mediawiki-l
> > >
> >
> > _______________________________________________
> > MediaWiki-l mailing list
> > MediaWiki-lWikimedia.org
> > http://mail.wikipedia.org/mailman/listinfo/mediawiki-l
> >
>
>
> --
> Kasimir Gabert
> _______________________________________________
> MediaWiki-l mailing list
> MediaWiki-lWikimedia.org
> http://mail.wikipedia.org/mailman/listinfo/mediawiki-l
>
_______________________________________________
MediaWiki-l mailing list
MediaWiki-lWikimedia.org
http://mail.wikipedia.org/mailman/listinfo/mediawiki-l

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Kelly Jones wrote:
> I'm trying to run mediawiki on a free server, but I'm
worried about
> bandwidth limits.

Ah, you want something for nothing. 

- -- brion vibber (brion  pobox.com)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org


iD8DBQFFmv+YwRnhpk1wk44RAvEUAJ9fjUsHmhdarPfSvqQrzZEzAGM5FQCg
3d+q
Io41t+UgTSQPdWSIKH0YF48=
=3KPj
-----END PGP SIGNATURE-----
_______________________________________________
MediaWiki-l mailing list
MediaWiki-lWikimedia.org
http://mail.wikipedia.org/mailman/listinfo/mediawiki-l

Kelly Jones wrote:
> I'm trying to run mediawiki on a free server, but I'm
worried about
> bandwidth limits.

Hosting is cheap. Unless you are working on a huge project
(and then 
you'll need money, money maker, and/or sponsors, but that's
the way it 
should be) it will cost you only a few bucks a month, top.


_______________________________________________
MediaWiki-l mailing list
MediaWiki-lWikimedia.org
http://mail.wikipedia.org/mailman/listinfo/mediawiki-l

On 1/2/07, Kelly Jones <kelly.terry.jonesgmail.com> wrote:
> The mirror servers should be as independent as
possible:
> geographically diverse, on different backbones, each
using their own
> MySQL server, owned by different people, etc. Some of
the mirror
> servers may be free, others may be paid hosting, others
may be
> dedicated servers, etc.

You're looking for the holy grail of hosting.

Some of this has been implemented via ideas like
mysql-cluster
(mentioned earlier) but I gather that mediawiki itself would
need to
be modified to support that technology.

Load-balancing a webserver is already something that's
fairly well known.


Basically you'd have to have some front-end computer which
understands
the bandwidth usage of all of its attached hosts, and it
would
intelligently balance the load over to those mirrors which
have
bandwidth left.

And then those secondary computers would each have an
installation
which would have some kind of synthronized database cache. 
This would
require a bit of MySQL magic, but it is supported by it. 
But it's not
supported by MediaWiki.. so edit collisions and such would
be
impossible to sort out if you have cached-reads from or
slow-writes
with the mirror sites..

But here's the problem.. what bandwidth is being saved, when
the same
copy of a page has to get synced to all of the other
mirrors?  Oh, you
could modify mediawiki to only update its local copy if a
page edit
request comes in and an old copy is stored locally.. but
where is the
disaster recovery when one of your mirrors goes down?

This is a complex issue.. and probably not worth the spare
change
which basic hosting would cost.  If the site is wildly
popular.. then
become self-supporting via memberships, ads, donations,
etc..
_______________________________________________
MediaWiki-l mailing list
MediaWiki-lWikimedia.org
http://mail.wikipedia.org/mailman/listinfo/mediawiki-l