lazy mirror / live backup

>> I'd like the ability to have gmirror do a more
efficient re-silvering (or
>> re-syncing) of the mirror members when a planned
disconnect occurs. This
>> would significantly reduce the mirror rebuild time
for any component which
>> had been deactivated, for network mirrors using
ggated devices this would
>> also reduce network usage and could be used for
remote asynced mirrors
>> thus providing a live backup for
laptops/workstations.

>Do you see that you are describing ZFS there? 
Considering the status of
>ZFS right now, I think it is not worth really trying to
implement such
>scheme in gmirror.

The problem with zfs is you cannot layer it as u can with
the geom 
classes.

For example if you want to create a failover zfs storage
pool, if you make 
the zfs pool out of gmirror devices with one being a local
device and the 
other being a ggatec device.  You would then have your zfs
raidz pool 
replicated on a remote host.  I do not think you can do this
with zfs by 
itself as you are not able to layer raidz pool ontop of a
load of zfs mirrors.

Mike.
_______________________________________________
freebsd-fsfreebsd.org mailing list

http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to
"freebsd-fs-unsubscribefreebsd.org"

According to Mike Wolman:
> The problem with zfs is you cannot layer it as u can
with the geom 
> classes.

Yes and no.  It is not as versatile as all geom classes I
believe but on
FreeBSD it is a geom provider.

> For example if you want to create a failover zfs
storage pool, if you make 
> the zfs pool out of gmirror devices with one being a
local device and the 
> other being a ggatec device.  You would then have your
zfs raidz pool 
> replicated on a remote host.  I do not think you can do
this with zfs by 
> itself as you are not able to layer raidz pool ontop of
a load of zfs 
> mirrors.

On plain zfs yes, you can have that.

zpool create tank raidz mirror da0 da1 mirror da2 da3 mirror
da4 da5.

(not tested)
-- 
Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=-
robertokeltia.freenix.fr
Darwin sidhe.keltia.net Kernel Version 8.8.2: Thu Sep 28
20:43:26 PDT 2006 i386
_______________________________________________
freebsd-fsfreebsd.org mailing list

http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to
"freebsd-fs-unsubscribefreebsd.org"

According to Mike Wolman:
>> The problem with zfs is you cannot layer it as u
can with the geom
>> classes.
>
>Yes and no.  It is not as versatile as all geom classes
I believe but on
>FreeBSD it is a geom provider.

I havent played with zfs that much and was unaware that zfs
created a 
device like the geom classes.

>> For example if you want to create a failover zfs
storage pool, if you make
>> the zfs pool out of gmirror devices with one being
a local device and the
>> other being a ggatec device.  You would then have
your zfs raidz pool
>> replicated on a remote host.  I do not think you
can do this with zfs by
>> itself as you are not able to layer raidz pool
ontop of a load of zfs
>> mirrors.
>
>On plain zfs yes, you can have that.
>
>zpool create tank raidz mirror da0 da1 mirror da2 da3
mirror da4 da5.

So could you do:

zpool create rank raidz mirror ad0 ggatec0 mirror ad1
ggatec1 mirror ad2 ggatec2

Then should the primary fileserver fail would the backup
machine be able 
to import this zfs pool?

Would you also be able to tune zfs to prefer the local disks
to the remote 
ones?

Mike.
_______________________________________________
freebsd-fsfreebsd.org mailing list

http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to
"freebsd-fs-unsubscribefreebsd.org"

On Sat, Apr 21, 2007 at 12:12:57AM +0100, Mike Wolman
wrote:
> 
> According to Mike Wolman:
> >>The problem with zfs is you cannot layer it as
u can with the geom
> >>classes.
> >
> >Yes and no.  It is not as versatile as all geom
classes I believe but on
> >FreeBSD it is a geom provider.
> 
> I havent played with zfs that much and was unaware that
zfs created a 
> device like the geom classes.
> 
> >>For example if you want to create a failover
zfs storage pool, if you make
> >>the zfs pool out of gmirror devices with one
being a local device and the
> >>other being a ggatec device.  You would then
have your zfs raidz pool
> >>replicated on a remote host.  I do not think
you can do this with zfs by
> >>itself as you are not able to layer raidz pool
ontop of a load of zfs
> >>mirrors.
> >
> >On plain zfs yes, you can have that.
> >
> >zpool create tank raidz mirror da0 da1 mirror da2
da3 mirror da4 da5.
> 
> So could you do:
> 
> zpool create rank raidz mirror ad0 ggatec0 mirror ad1
ggatec1 mirror ad2 
> ggatec2

AFAIK you can't do raidz over mirror, unless you mirror at
disk layer.
Just mirror would work for shure.

> Then should the primary fileserver fail would the
backup machine be able 
> to import this zfs pool?

This is obviously the primary reason for such a mirror, but
it also the
most insane point about it.
While you can import it on another machine you completely
loose the state
information, since the backup side is modified in an unknown
way.
That's the case with your gmirror enhancements as well.
You completely trust the sync state of an offline disk.

> Would you also be able to tune zfs to prefer the local
disks to the remote 
> ones?

No, unless you offline them.

But the point with ZFS is that you don't neeed such mirror
hacks at all.
Just create a pool for your data and one or more for
backup.
The backup pools can be on different machines as well - no
need to
ggate the disks.
Then setup a cron-job to build snapshots on short time
regular base.
"zfs send" it into the "zfs receive" on
the backup host.
E.g. by using ssh.
This can be done differentially between two snapshots, no
need to do
a full transfer everytime.
In case you need to switch to the backup side, you can
easily rool back
any side you want to get back in sync.
You can also just replace the disks on your primary system
with the
disks from the backup and just import the pool.
This even works for a slow link, because writers don't need
to wait
for the backup system to keep up, wich would be the case for
ggate
based mirrors over slow links.
And if you still want to disconnect/reconnect drives on the
main system
you can zpool import/export your backup pool disks and run
the
zfs send/receive localy.
There is no need that the backup disks are of the same size,
because
you copy based on data not disk blocks.
E.g. your backup pool can be much bigger than the primary
and keep
more snapshots, or it is smaller and keep less snapshots.
You can use cheap SATA disks to backup a SCSI system.
You can compress the data on the backup pool, while not
using
compression on the primary pool - or use stronger
compression.

-- 
B.Walter                http://www.bwct.de      http://www.fizon.de
berndbwct.de           infobwct.de           
supportfizon.de
_______________________________________________
freebsd-fsfreebsd.org mailing list

http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to
"freebsd-fs-unsubscribefreebsd.org"

>>>> For example if you want to create a
failover zfs storage pool, if you make
>>>> the zfs pool out of gmirror devices with
one being a local device and the
>>>> other being a ggatec device.  You would
then have your zfs raidz pool
>>>> replicated on a remote host.  I do not
think you can do this with zfs by
>>>> itself as you are not able to layer raidz
pool ontop of a load of zfs
>>>> mirrors.
>>>
>>> On plain zfs yes, you can have that.
>>>
>>> zpool create tank raidz mirror da0 da1 mirror
da2 da3 mirror da4 da5.
>>
>> So could you do:
>>
>> zpool create rank raidz mirror ad0 ggatec0 mirror
ad1 ggatec1 mirror ad2
>> ggatec2
>
> AFAIK you can't do raidz over mirror, unless you mirror
at disk layer.
> Just mirror would work for shure.
>
>> Then should the primary fileserver fail would the
backup machine be able
>> to import this zfs pool?
>
> This is obviously the primary reason for such a mirror,
but it also the
> most insane point about it.
> While you can import it on another machine you
completely loose the state
> information, since the backup side is modified in an
unknown way.
> That's the case with your gmirror enhancements as
well.
> You completely trust the sync state of an offline
disk.
>
>> Would you also be able to tune zfs to prefer the
local disks to the remote
>> ones?
>
> No, unless you offline them.
>
> But the point with ZFS is that you don't neeed such
mirror hacks at all.
> Just create a pool for your data and one or more for
backup.
> The backup pools can be on different machines as well -
no need to
> ggate the disks.
> Then setup a cron-job to build snapshots on short time
regular base.
> "zfs send" it into the "zfs
receive" on the backup host.
> E.g. by using ssh.
> This can be done differentially between two snapshots,
no need to do
> a full transfer everytime.
> In case you need to switch to the backup side, you can
easily rool back
> any side you want to get back in sync.
> You can also just replace the disks on your primary
system with the
> disks from the backup and just import the pool.
> This even works for a slow link, because writers don't
need to wait
> for the backup system to keep up, wich would be the
case for ggate
> based mirrors over slow links.
> And if you still want to disconnect/reconnect drives on
the main system
> you can zpool import/export your backup pool disks and
run the
> zfs send/receive localy.
> There is no need that the backup disks are of the same
size, because
> you copy based on data not disk blocks.
> E.g. your backup pool can be much bigger than the
primary and keep
> more snapshots, or it is smaller and keep less
snapshots.
> You can use cheap SATA disks to backup a SCSI system.
> You can compress the data on the backup pool, while not
using
> compression on the primary pool - or use stronger
compression.
>

Thank you for enlightening me on the zfs send and receive
stuff, for the 
above example i can see how well suited zfs is.

My only concern is zfs is quite heavy weight (memory wise)
compared to 
gmirror and ufs for a simple laptop/desktop setup which you
may just want 
to replicate the entire drive and grab the ggatec device and
be ready to 
run should anything happen to the machine.

Im sure there are other situations where running zfs on the
available 
hardware is not an option compared to gmirror - im not sure
what the 
recommended amount for freebsd but as far as i can rember
the suggested 
about for solaris is 1Gb - comparing to gmirror i think i
have run it on a 
machine doing simple home fileserving with 128Mb.

Mike.
_______________________________________________
freebsd-fsfreebsd.org mailing list

http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to
"freebsd-fs-unsubscribefreebsd.org"

On Sat, Apr 21, 2007 at 01:38:31AM +0100, Mike Wolman
wrote:
> My only concern is zfs is quite heavy weight (memory
wise) compared to 
> gmirror and ufs for a simple laptop/desktop setup which
you may just want 
> to replicate the entire drive and grab the ggatec
device and be ready to 
> run should anything happen to the machine.

Yes - that's unfortunately true.

> Im sure there are other situations where running zfs on
the available 
> hardware is not an option compared to gmirror - im not
sure what the 
> recommended amount for freebsd but as far as i can
rember the suggested 
> about for solaris is 1Gb - comparing to gmirror i think
i have run it on a 
> machine doing simple home fileserving with 128Mb.

Ever thought about UFS snapshots backed with rsync?
You get a consitent pseudo image of your running filesystem
with
unallocated blocks represented as zeros.
rsync now allows comparing file chunks and copies only
differences.
Still every block need to be read, though.

vbackup from devel/plan9port stores checksums and allows
offering
only different blocks to the other side.
venti - the backing store behind vbackup - allows
compression and
single storage of different blocks with same data, which
reduces
the required backup capacity very impressive.
My expirience with vbackup is that this mechanism is fast
enough
as long as there are no hughe differences.

-- 
B.Walter                http://www.bwct.de      http://www.fizon.de
berndbwct.de           infobwct.de           
supportfizon.de
_______________________________________________
freebsd-fsfreebsd.org mailing list

http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to
"freebsd-fs-unsubscribefreebsd.org"

On Sat, 21 Apr 2007, Bernd Walter wrote:

> On Sat, Apr 21, 2007 at 01:38:31AM +0100, Mike Wolman
wrote:
>> My only concern is zfs is quite heavy weight
(memory wise) compared to
>> gmirror and ufs for a simple laptop/desktop setup
which you may just want
>> to replicate the entire drive and grab the ggatec
device and be ready to
>> run should anything happen to the machine.
>
> Yes - that's unfortunately true.
>
>> Im sure there are other situations where running
zfs on the available
>> hardware is not an option compared to gmirror - im
not sure what the
>> recommended amount for freebsd but as far as i can
rember the suggested
>> about for solaris is 1Gb - comparing to gmirror i
think i have run it on a
>> machine doing simple home fileserving with 128Mb.
>
> Ever thought about UFS snapshots backed with rsync?
> You get a consitent pseudo image of your running
filesystem with
> unallocated blocks represented as zeros.
> rsync now allows comparing file chunks and copies only
differences.
> Still every block need to be read, though.

Yea i use rsync and snapshots quite a bit, but unfortunately
rsync works 
at the filesystem level so you cant really get a bootable
image of the whole 
device.  It would be nice if this could be done without user
interaction, 
ie if the ggatec component of a mirror disappears and
reappears gmirror 
justs gets to work syncing things up.

>
> vbackup from devel/plan9port stores checksums and
allows offering
> only different blocks to the other side.
> venti - the backing store behind vbackup - allows
compression and
> single storage of different blocks with same data,
which reduces
> the required backup capacity very impressive.
> My expirience with vbackup is that this mechanism is
fast enough
> as long as there are no hughe differences.
>

I'll take a peek as it sounds interesting,

Mike.

_______________________________________________
freebsd-fsfreebsd.org mailing list

http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to
"freebsd-fs-unsubscribefreebsd.org"

On Fri, Apr 20, 2007 at 09:04:11PM -0500, Eric Anderson
wrote:
> On 04/20/07 20:25, Mike Wolman wrote:
> >
> >On Sat, 21 Apr 2007, Bernd Walter wrote:
> >
> >>On Sat, Apr 21, 2007 at 01:38:31AM +0100, Mike
Wolman wrote:
> >
> >Yea i use rsync and snapshots quite a bit, but
unfortunately rsync works 
> >at the filesystem level so you cant really get a
bootable image of the 
> >whole device.  It would be nice if this could be
done without user 
> >interaction, ie if the ggatec component of a mirror
disappears and 
> >reappears gmirror justs gets to work syncing things
up.
> 
> rsync and it's like are great tools, but traversing the
tree every time 
> is not only slow and heavy handed, but it takes an
enormous amount of 
> memory (akin to fsck) for large file systems.  rsync
isn't the right 
> tool for lazy syncing.

Thich file tree?
I was talking about a snapshot, which is a single file.

-- 
B.Walter                http://www.bwct.de      http://www.fizon.de
berndbwct.de           infobwct.de           
supportfizon.de
_______________________________________________
freebsd-fsfreebsd.org mailing list

http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to
"freebsd-fs-unsubscribefreebsd.org"

On 04/21/07 05:02, Bernd Walter wrote:
> On Fri, Apr 20, 2007 at 09:04:11PM -0500, Eric Anderson
wrote:
>> On 04/20/07 20:25, Mike Wolman wrote:
>>> On Sat, 21 Apr 2007, Bernd Walter wrote:
>>>
>>>> On Sat, Apr 21, 2007 at 01:38:31AM +0100,
Mike Wolman wrote:
>>> Yea i use rsync and snapshots quite a bit, but
unfortunately rsync works 
>>> at the filesystem level so you cant really get
a bootable image of the 
>>> whole device.  It would be nice if this could
be done without user 
>>> interaction, ie if the ggatec component of a
mirror disappears and 
>>> reappears gmirror justs gets to work syncing
things up.
>> rsync and it's like are great tools, but traversing
the tree every time 
>> is not only slow and heavy handed, but it takes an
enormous amount of 
>> memory (akin to fsck) for large file systems. 
rsync isn't the right 
>> tool for lazy syncing.
> 
> Thich file tree?
> I was talking about a snapshot, which is a single
file.
> 


Oh, I didn't realize you were recommending rsync'ing the
snapshot file 
itself.  Wow.  That seems pretty brutal of a solution to me.
 The only 
gain is the reduced data transferred over the net, and not
the quicker 
sync time, less load on your source disks, etc.

I think everyone knows that there are some solutions to the
problem. 
The problem is they are not good or elegant solutions.  What
Mike is 
proposing is an elegant solution, that is slick and smooth,
and simple.

Eric


_______________________________________________
freebsd-fsfreebsd.org mailing list

http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to
"freebsd-fs-unsubscribefreebsd.org"