keeping instances of bric in sync

Hello,

I'm trying to set up an environment of 2 instances of bric,
such that one 
can continue to be used in production, and the other can be
used to 
implement a redesign.  I've set up the second bric instance
using a clone, 
and it works.

What I'd like to do now is (nightly) update all the stories
and media from 
the prod to the dev, while leaving the templates untouched. 
This way, 
when the time comes to flip the switch, it will all be in
place, with 
current content and brand new templates.

I'd guess that *someone* must be doing something like
this...

I intended to use bric_soap for this:
bric_soap story list_ids --server http://bric/ |bric_soap story export

--server http://bric/ - |
bric_soap story update --server 
http://bric:8080/

I am, however, running into a snag with unicode:
Call to Bric::SOAP::Story->export failed:

Code point u0001 is not a valid character in XML at 
/usr/lib/perl5/site_perl/5.8.5/Bric/SOAP/Util.pm line 658


So, perhaps I'm hoping for too much from bric_soap, and
instead should 
build a more custom script for my needs.  Two issues:

* is this the correct approach to keeping content in sync? 
If not, what 
would be better.
* How can I work around this invalid character?

Thanks in advance!

John

On 5-Dec-06, at 5:45 PM, johngrumpet.net wrote:

> I'm trying to set up an environment of 2 instances of
bric, such  
> that one can continue to be used in production, and the
other can  
> be used to implement a redesign.

Gosh: I've always held that this was one of Bricolage's
major  
strengths -- not having to have separate development and
production  
systems during a major re-design -- as all of the necessary
pieces in  
in place to do it side-by-side on the same installation.
Hopefully  
I'm not the only one who thinks so?

--
Phillip Smith,
Simplifier of Technology
Community Bandwidth

>> I'm trying to set up an environment of 2 instances
of bric, such that one 
>> can continue to be used in production, and the
other can be used to 
>> implement a redesign.
>
> Gosh: I've always held that this was one of Bricolage's
major strengths -- 
> not having to have separate development and production
systems during a major 
> re-design -- as all of the necessary pieces in in place
to do it side-by-side 
> on the same installation. Hopefully I'm not the only
one who thinks so?
>

I think this is probably generally true, but users want to
continue to be 
able to preview their edits in the context of the current
site.  If 
template developers start messing with templates, even if
they don't 
deploy them, it's going to create some confusion.  And then
there's a 
political element: what if the web team doesn't want to
entertain comments 
about the redesign until it's done?  We need to maintain a
non-disruptive 
environment for web maintenance.

I'd agree more if it were just me and a handful of others
working on it.

Thanks,

John

johngrumpet.net writes:
> I'm trying to set up an environment of 2 instances of
bric, such that
> one can continue to be used in production, and the
other can be used
> to implement a redesign.  
> [...]
> I'd guess that *someone* must be doing something like
this...

Just for the discussion, I need to build a similar
infrastructure (but
haven't done it yet) and I think it's indeed a typical
demand.


> /usr/lib/perl5/site_perl/5.8.5/Bric/SOAP/Util.pm line
658
                           ^^^^^ 

Just a distant idea: Maybe you try a newer Perl, eg. 5.8.8.
Aren't
many bugfixes in Perl 5.8.x around Unicode?

Steffen 
-- 
Steffen Schwigon <schwigonwebit.de>
Dresden Perl Mongers <http://dresden-pm.org/>

Deutscher Perl-Workshop <http://www.perl-work
shop.de/>

On Dec 5, 2006, at 14:45, johngrumpet.net wrote:

> I intended to use bric_soap for this:
> bric_soap story list_ids --server http://bric/ |bric_soap story  
> export --server http://bric/
- | bric_soap story update --server  
> http://bric:8080/
>
> I am, however, running into a snag with unicode:
> Call to Bric::SOAP::Story->export failed:
>
> Code point u0001 is not a valid character in XML at
/usr/lib/perl5/ 
> site_perl/5.8.5/Bric/SOAP/Util.pm line 658

What version of Bricolage? Is it the import or export server
that  
gives this error? What's at line 658 in Bric::SOAP::Util?

> So, perhaps I'm hoping for too much from bric_soap, and
instead  
> should build a more custom script for my needs.  Two
issues:
>
> * is this the correct approach to keeping content in
sync?  If not,  
> what would be better.

Well, yes, it's correct, as long as you don't have too many
stories.  
If you have a lot, updating them all every night will be
incredibly  
slow.

> * How can I work around this invalid character?

That is the question!

Best,

David

On Dec 5, 2006, at 23:58, Steffen Schwigon wrote:

>> /usr/lib/perl5/site_perl/5.8.5/Bric/SOAP/Util.pm
line 658
>                            ^^^^^
>
> Just a distant idea: Maybe you try a newer Perl, eg.
5.8.8. Aren't
> many bugfixes in Perl 5.8.x around Unicode?

Yes, but it should be pretty solid from 5.8.3 on.

Best,

David

Hi David,

>> Code point u0001 is not a valid character in XML
at 
>> /usr/lib/perl5/site_perl/5.8.5/Bric/SOAP/Util.pm
line 658
>
> What version of Bricolage? Is it the import or export
server that gives this 
> error? What's at line 658 in Bric::SOAP::Util?

* 1.10.2
* export server
* if (defined $data and length $data) {

>> * is this the correct approach to keeping content
in sync?  If not, what 
>> would be better.
>
> Well, yes, it's correct, as long as you don't have too
many stories. If you 
> have a lot, updating them all every night will be
incredibly slow.

What's alot?  We have, currently...
bric ~ % bric_soap story list_ids --server http://bric/ | wc -l
    2673
bric ~ % bric_soap media list_ids --server http://bric/ | wc -l
    4110
bric ~ %

>
>> * How can I work around this invalid character?
>
> That is the question!

I like Steffen's suggestion about upgrading Perl. 
Unfortunately, easier 
said than done.  I think I know a way to test this theory,
going to try 
that next.

Thanks,

John

On Dec 6, 2006, at 10:17, johngrumpet.net wrote:

> * 1.10.2
> * export server
> * if (defined $data and length $data) {

Hrm. Well somehow you have invalid unicode somewhere. Try
printing  
out the offending line (by catching the exception inside an
eval) and  
see what you get. Odd that it would be broken, though, given
that it  
comes from the database.

> What's alot?  We have, currently...
> bric ~ % bric_soap story list_ids --server http://bric/ | wc -l
>    2673
> bric ~ % bric_soap media list_ids --server http://bric/ | wc -l
>    4110
> bric ~ %

I'd say that's a lot. It will take quite a while to sync
those.  
Better would be to update only those that have changed since
the last  
time you updated. You'd have to query for the last time a
story was  
checked in.

> I like Steffen's suggestion about upgrading Perl. 
Unfortunately,  
> easier said than done.  I think I know a way to test
this theory,  
> going to try that next.

Good luck!

Best,

David

>
>> * 1.10.2
>> * export server
>> * if (defined $data and length $data) {
>
> Hrm. Well somehow you have invalid unicode somewhere.
Try printing out the 
> offending line (by catching the exception inside an
eval) and see what you 
> get. Odd that it would be broken, though, given that it
comes from the 
> database.

Good guess.  I did:
for i in `bric_soap story list_ids --server http://bric/`; do echo 
"exporting story $i"; bric_soap story export
--server http://bric/ $i; 
done
and grepped for the error.  Found the offending story, fixed
it - it was a 
relic from a couple of years ago - and off I went.

>> bric ~ % bric_soap story list_ids --server http://bric/ | wc -l
>>   2673
>> bric ~ % bric_soap media list_ids --server http://bric/ | wc -l
>>   4110
>
> I'd say that's a lot. It will take quite a while to
sync those. Better would 
> be to update only those that have changed since the
last time you updated. 
> You'd have to query for the last time a story was
checked in.

Excellent suggestion.  This seems to work:
for i in `bric_soap story list_ids --server http://bric/ --search 
"publish_date_start=2006-12-05T00:00:00Z"`; do
echo "exporting $i"; 
bric_soap story export --server http://bric/ $i |bric_soap story
update 
--server http://bric:8080/ -
; done

Works for media as well.

Thanks, as usual!

John