Strange error on log transport / data guard, SLES10

Running SLES10 / UL4.4 oracle data guard tests, see strange
errro when running primary database on SuSe side and 
create high transactional load:

Errors in file
/opt/oracle/admin/testdb/bdump/testdb_lgwr_12231.trc:
ORA-03113: end-of-file on communiLGWR: Network asynch I/O
wait error 3113 log 3 service
'(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=tes
trac14)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=testdbst_XPT
)(INSTANCE_NAME=testdbst)(SERVER=dedicated)))'
Wed Apr 11 15:57:51 2007


Any idea what it can be about? When running from UL4.4 to
SLES10, I don't see any of these errors.

HI ALEXEI,

ARE THE PING TIMES AT THIS MOMENT OKAY OR IS THERE A
NETWORKING ISSUE? DO YOU 
EXPERIENCE THIS ERROR WITH OPTION ARCH INSTEAD OF LGWR, TOO?
(JUST FOR 
TESTING OF COURSE).

AM DONNERSTAG 12 APRIL 2007 01:00 SCHRIEB ALEXEI_ROUDNEV:
> RUNNING SLES10 / UL4.4 ORACLE DATA GUARD TESTS, SEE
STRANGE ERRRO WHEN
> RUNNING PRIMARY DATABASE ON SUSE SIDE AND CREATE HIGH
TRANSACTIONAL LOAD:
>
> ERRORS IN FILE
/OPT/ORACLE/ADMIN/TESTDB/BDUMP/TESTDB_LGWR_12231.TRC:
> ORA-03113: END-OF-FILE ON COMMUNILGWR: NETWORK ASYNCH
I/O WAIT ERROR 3113
> LOG 3 SERVICE
>
'(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=TES
TRAC14)(PORT=15
>21)))(CONNECT_DATA=(SERVICE_NAME=TESTDBST_XPT)(INSTANCE_
NAME=TESTDBST)(SERVE
>R=DEDICATED)))' WED APR 11 15:57:51 2007
>
>
> ANY IDEA WHAT IT CAN BE ABOUT? WHEN RUNNING FROM UL4.4
TO SLES10, I DON'T
> SEE ANY OF THESE ERRORS.

-- 
FREUNDLICHE GRüßE

I.A.
MARTIN KLIER
SYSTEMADMINISTRATION/DATENBANKEN
------------------------------------------------------------
------
A.T.U AUTO-TEILE-UNGER
HANDELS GMBH & CO. KG

I tested connection - works pretty well, FTP speed is
35MB/second in both
directions (then limited by the File System).

It can be UL4.4 problem (on receiving side), but it looks
more likely as
Async IO problem in SLES10 - Oracle 10.2.0.3 uses
async io when sending logs in SYNC mode (I run MAx
Availiability mode + Fast
Failover in the lab), and somethinh runs out of the limits.

Will retest on SLES9 <->SLES9 in a short time.

(To Arun - when running Data Guard between SLES10 and UL4.4,
both with
Oracle 10.2.0.3 and x86_64,
when I run SLES10 as primary and put a heavy load on it, log
transport fail.
I posted error messages some time ago, looks as it fail
somewhere in Async
IO on the network. NO any error messages seen on UL4.4 side.
When primary is
on UL4.4, everything works fine and
logs are transported in time /REDO APPLY is little slower so
I have a delay
in redo apply - about 2 - 3 logs when I finish test pass,
but connections do
not fail/).




----- Original Message ----- 
From: "Martin Klier" <martin.klieratu.de>
To: <suse-oraclesuse.com>
Cc: "Alexei_Roudnev" <Alexei_Roudnevexigengroup.com>
Sent: Thursday, April 12, 2007 1:44 AM
Subject: Re: [suse-oracle] Strange error on log transport /
data guard,
SLES10



-- 
To unsubscribe, email: suse-oracle-unsubscribesuse.com
For additional commands, email: suse-oracle-helpsuse.com
Please see http://www.suse.com/oracl
e/ before posting

We've had an open TAR on a corrupt archive logs on dataguard
issue with
Oracle for two years. The last update to the TAR just a week
ago said
that Oracle development could reproduce and solve the
problem by setting
filesystemio_options = setall or directio instead of asynch.
Only
problem with that is you must create ALL of your original
data files
with directio in the first place, so we are stuck.

The problems we see are much more subtle, just a message
about a corrupt
log on the standby. The standby eventually recovers, but the
result is
that many archive logs must be shipped to the standby
multiple times.
That just makes things worse and slower. At one point we
could reproduce
on test equipment 100% of the time. We were clobbered with
this issue
after an SLES8 kernel upgrade and an Oracle upgrade so we
stopped
upgrading.

We have given up trying to solve the issue and we'll stick
with what
we've got until our next hardware refresh as we have reached
an
"understanding" with our hardware and software
combinations. Sad really.

Best of luck solving this issue.

Andy

> -----Original Message-----
> From: Alexei_Roudnev [mailto:Alexei_Roudnevexigengroup.com] 
> Sent: Monday, April 16, 2007 10:25 PM
> To: suse-oraclesuse.com; Martin Klier
> Cc: Arun Singh
> Subject: Re: [suse-oracle] Strange error on log
transport / 
> data guard, SLES10
> 
> I tested connection - works pretty well, FTP speed is 
> 35MB/second in both
> directions (then limited by the File System).
> 
> It can be UL4.4 problem (on receiving side), but it
looks 
> more likely as
> Async IO problem in SLES10 - Oracle 10.2.0.3 uses
> async io when sending logs in SYNC mode (I run MAx 
> Availiability mode + Fast
> Failover in the lab), and somethinh runs out of the
limits.
> 
> Will retest on SLES9 <->SLES9 in a short time.
> 
snip

--
To unsubscribe, email: suse-oracle-unsubscribesuse.com
For additional commands, email: suse-oracle-helpsuse.com
Please see http://www.suse.com/oracl
e/ before posting

1) I don't think that ''you must create ALL of your original
data files
with directio in the first place", directio can be
changed without changing
data.

2) Looks as turning async io on for FS trigger some bug.

3) But when I run primary on UL4.4 (and standby on SLES10) I
dion't have
such problem.

5) Both have 'setall' setting, SLES10 have reiserfs +
Notail, and UL4.4 have
ext3.

6) Archive shipping mode is SYNC (or what is required for
Fast Failover?
SYNC, as I remember).


----- Original Message ----- 
From: "McAllister, Andrew" <McAllisterAumsystem.edu>
To: "Alexei_Roudnev" <Alexei_Roudnevexigengroup.com>;
<suse-oraclesuse.com>; "Martin Klier"
<martin.klieratu.de>
Cc: "Arun Singh" <Arun.Singhnovell.com>
Sent: Tuesday, April 17, 2007 6:22 AM
Subject: RE: [suse-oracle] Strange error on log transport /
data guard,
SLES10


We've had an open TAR on a corrupt archive logs on dataguard
issue with
Oracle for two years. The last update to the TAR just a week
ago said
that Oracle development could reproduce and solve the
problem by setting
filesystemio_options = setall or directio instead of asynch.
Only
problem with that is you must create ALL of your original
data files
with directio in the first place, so we are stuck.

The problems we see are much more subtle, just a message
about a corrupt
log on the standby. The standby eventually recovers, but the
result is
that many archive logs must be shipped to the standby
multiple times.
That just makes things worse and slower. At one point we
could reproduce
on test equipment 100% of the time. We were clobbered with
this issue
after an SLES8 kernel upgrade and an Oracle upgrade so we
stopped
upgrading.

We have given up trying to solve the issue and we'll stick
with what
we've got until our next hardware refresh as we have reached
an
"understanding" with our hardware and software
combinations. Sad really.

Best of luck solving this issue.

Andy

> -----Original Message-----
> From: Alexei_Roudnev [mailto:Alexei_Roudnevexigengroup.com]
> Sent: Monday, April 16, 2007 10:25 PM
> To: suse-oraclesuse.com; Martin Klier
> Cc: Arun Singh
> Subject: Re: [suse-oracle] Strange error on log
transport /
> data guard, SLES10
>
> I tested connection - works pretty well, FTP speed is
> 35MB/second in both
> directions (then limited by the File System).
>
> It can be UL4.4 problem (on receiving side), but it
looks
> more likely as
> Async IO problem in SLES10 - Oracle 10.2.0.3 uses
> async io when sending logs in SYNC mode (I run MAx
> Availiability mode + Fast
> Failover in the lab), and somethinh runs out of the
limits.
>
> Will retest on SLES9 <->SLES9 in a short time.
>
snip


-- 
To unsubscribe, email: suse-oracle-unsubscribesuse.com
For additional commands, email: suse-oracle-helpsuse.com
Please see http://www.suse.com/oracl
e/ before posting