|
List Info
Thread: Re: server hanging
|
|
| Re: server hanging |
 
Australia |
2007-04-29 00:45:58 |
Ahh yep i get a tonne of these thrown
if (!(message instanceof IStreamData)) {
throw new RuntimeException(
"expected IStreamData but got
" +
message.getClass() + " (type " +
message.getDataType() + ")");
}
something up there.
Steven Gong wrote:
> Don't know if it helps. Try to add a line:
> if (message == null) return false;
> at the beginning of okayToSendMessage() in
PlaylistSubscriberStream.
>
> On 4/29/07, *Dan Rossi* <spam electroteque.org
> <mailto:spamelectroteque.org>> wrote:
>
> OK, which file / line ?
>
> Steven Gong wrote:
> > It's easy to make a workaround for this in
okayToSendMessage() to
> > ignore the null pointer but I think we'd
better find the root cause
> > this issue as this should be happen in normal
cases.
> >
> > On 4/29/07, *Dan Rossi* <spamelectroteque.org
> <mailto:spamelectroteque.org>
> > <mailto:spamelectroteque.org
<mailto:spamelectroteque.org>>>
> wrote:
> >
> > OK it did it again, hanged without
warnings at all this time
> until i
> > tried to stop the service and then
displayed JVM dumps,
> logged via the
> > java service wrapper not red5. Ive put
red5 logging into
> fatal to
> > see if
> > that helps ?
> >
> > Steven Gong wrote:
> > > Will this bug hang the entire server?
I guess this should be a
> > > contention that the pengingMessage
has been released
> before it's
> > > pushed to the consumer. So the body
of pendingMessage is
> null and
> > > causes that exception.
> > >
> > > Dan, if possible, please record this
as a tracker so that
> this won't
> > > be forgotten. Thanks.
> > >
> > > On 4/29/07, *Dan Rossi* < spamelectroteque.org
> <mailto:spamelectroteque.org>
> > <mailto:spamelectroteque.org
<mailto:spamelectroteque.org>>
> > > <mailto: spamelectroteque.org
> <mailto:spamelectroteque.org>
<mailto:spamelectroteque.org
> <mailto:spamelectroteque.org>>>>
> > wrote:
> > >
> > > I already have for the exception
errors should i
> duplicate the
> > > report ?
> > >
> > > Thijs Triemstra | Collab wrote:
> > > > Please create a ticket on
JIRA..
> > > >
> > > > cheers,
> > > >
> > > > Thijs
> > > >
> > > > On Apr 29, 2007, at 2:43 AM,
Dan Rossi wrote:
> > > >
> > > >
> > > >> Hi this is a bizarre one
but after quite a while
> having
> > the server
> > > >> up it
> > > >> will hang without any
outofmemory error blocking
> > connections it
> > > could
> > > >> because of these kinds
of exceptions
> > > >>
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | [ERROR]
> 745875
> > > >> pool-3-thread-2:
> > > >> (
>
org.red5.server.stream.PlaylistSubscriberStream.error
> > > >> ) Error while pulling
message.
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 |
> > > >>
java.lang.NullPointerException
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
org.red5.server.stream.PlaylistSubscriberStream
> > > >>
> >
>
$PlayEngine.okayToSendMessage(PlaylistSubscriberStream.java:
1241)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
org.red5.server.stream.PlaylistSubscriberStream
> > > >>
>
$PlayEngine.pullAndPush(PlaylistSubscriberStream.java:1301)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
> >
>
org.red5.server.stream.PlaylistSubscriberStream$PlayEngine.a
ccess
> > > >>
$100(PlaylistSubscriberStream.java:682)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
org.red5.server.stream.PlaylistSubscriberStream.written
> > > >>
(PlaylistSubscriberStream.java :535)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
org.red5.server.net.rtmp.BaseRTMPHandler.messageSent
> > > >>
(BaseRTMPHandler.java:205)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
org.red5.server.net.rtmp.RTMPMinaIoHandler.messageSent
> > > >>
(RTMPMinaIoHandler.java:169)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
org.apache.mina.common.support.AbstractIoFilterChain
> > > >>
$TailFilter.messageSent(
> AbstractIoFilterChain.java:716)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
> > >
> >
>
org.apache.mina.common.support.AbstractIoFilterChain.callNex
tMessageSe
> > > >> nt(
AbstractIoFilterChain.java:391)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
> >
org.apache.mina.common.support.AbstractIoFilterChain.access$
1300
> > > >>
(AbstractIoFilterChain.java :54)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
> >
org.apache.mina.common.support.AbstractIoFilterChain$EntryIm
pl
> > > >>
$1.messageSent(AbstractIoFilterChain.java :806)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
org.red5.io.filter.ExecutorFilter.processEvent
> > > (ExecutorFilter.java:
> > > >> 358)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
>
org.red5.io.filter.ExecutorFilter$ProcessEventsRunnable.run
> > > >>
(ExecutorFilter.java:416)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
java.util.concurrent.ThreadPoolExecutor$Worker.runTask
> > (Unknown
> > > Source)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
> >
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
java.lang.Thread.run(Unknown Source)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | [ERROR]
> 745875
> > > >> pool-3-thread-2:
> > > >> (
org.red5.server.stream.PlaylistSubscriberStream.error
> > > >> ) Error while pulling
message.
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 |
> > > java.lang.RuntimeException:
> > > >> expected IStreamData but
got class
> > > >>
org.red5.server.net.rtmp.event.Unknown (type 91)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
org.red5.server.stream.PlaylistSubscriberStream
> > > >>
$PlayEngine.okayToSendMessage(
> > PlaylistSubscriberStream.java:1203)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
org.red5.server.stream.PlaylistSubscriberStream
> > > >>
>
$PlayEngine.pullAndPush(PlaylistSubscriberStream.java
:1301)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
> >
>
org.red5.server.stream.PlaylistSubscriberStream$PlayEngine.a
ccess
> > > >>
$100(PlaylistSubscriberStream.java:682)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
>
org.red5.server.stream.PlaylistSubscriberStream.written
> > > >>
(PlaylistSubscriberStream.java:535)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
org.red5.server.net.rtmp.BaseRTMPHandler.messageSent
> > > >>
(BaseRTMPHandler.java:205)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
org.red5.server.net.rtmp.RTMPMinaIoHandler.messageSent
> > > >>
(RTMPMinaIoHandler.java:169)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
org.apache.mina.common.support.AbstractIoFilterChain
> > > >>
$TailFilter.messageSent(
> AbstractIoFilterChain.java:716)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
> > >
> >
>
org.apache.mina.common.support.AbstractIoFilterChain.callNex
tMessageSe
> > >
> > > >> nt(
AbstractIoFilterChain.java:391)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
> >
org.apache.mina.common.support.AbstractIoFilterChain.access$
1300
> > > >>
(AbstractIoFilterChain.java :54)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
> >
org.apache.mina.common.support.AbstractIoFilterChain$EntryIm
pl
> > > >>
$1.messageSent(AbstractIoFilterChain.java :806)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
> >
>
org.red5.io.filter.ExecutorFilter.processEvent(ExecutorFilte
r.java:
> > > >> 358)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
>
org.red5.io.filter.ExecutorFilter$ProcessEventsRunnable.run
> > > >>
(ExecutorFilter.java:416)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
java.util.concurrent.ThreadPoolExecutor$Worker.runTask
> > (Unknown
> > > Source)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
> >
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
> > > >>
java.lang.Thread.run(Unknown Source)
> > > >> INFO | jvm 1 |
2007/04/28 14:01:52 | [ERROR]
> 1062437
> > > >>
DefaultQuartzScheduler_Worker-8:(
> > > >>
>
org.red5.server.scheduling.QuartzSchedulingService.error
> > ) Job
> > > >>
>
org.red5.server.net.rtmp.RTMPConnection$KeepAliveJobd0570e
> > > execution
> > > >> failed: null
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | [ERROR]
> 1083125
> > > >> pool-3-thread-6:(
> >
org.red5.server.service.ServiceInvoker.error
> > > ) Error
> > > >> executing call: Service:
null Method: closeStream Num
> > Params: 0
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | [ERROR]
> 1083141
> > > >> pool-3-thread-6:(
> >
org.red5.server.service.ServiceInvoker.error )
> > > >> Service
> > > >> invocation error
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 |
> > > >>
java.lang.reflect.InvocationTargetException
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
> > > >>
>
sun.reflect.GeneratedMethodAccessor21.invoke(Unknown
Source)
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
> > > >>
sun.reflect.DelegatingMethodAccessorImpl.invoke
> (Unknown
> > Source)
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
> > > >>
java.lang.reflect.Method.invoke(Unknown Source)
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
> > > >>
> > >
> >
>
org.red5.server.service.ServiceInvoker.invoke(ServiceInvoker
.java:181)
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
> > > >>
> > >
> >
>
org.red5.server.net.rtmp.RTMPHandler.invokeCall(RTMPHandler.
java :174)
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
> > > >>
org.red5.server.net.rtmp.RTMPHandler.onInvoke
> > (RTMPHandler.java:355)
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
> > > >>
>
org.red5.server.net.rtmp.BaseRTMPHandler.messageReceived
> > > >> (
BaseRTMPHandler.java:126)
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
> > > >>
>
org.red5.server.net.rtmp.RTMPMinaIoHandler.messageReceived
> > > >> (
RTMPMinaIoHandler.java:116)
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
> > > >>
org.apache.mina.common.support.AbstractIoFilterChain
> > > >>
$TailFilter.messageReceived(
> AbstractIoFilterChain.java:703)
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
> > > >>
> > >
> >
>
org.apache.mina.common.support.AbstractIoFilterChain.callNex
tMessageRe
>
> > >
> > > >>
ceived(AbstractIoFilterChain.java:362)
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
> > > >>
> >
>
org.apache.mina.common.support.AbstractIoFilterChain.access$
1200
> > > >>
(AbstractIoFilterChain.java :54)
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
> > > >>
> >
org.apache.mina.common.support.AbstractIoFilterChain$EntryIm
pl
> > > >>
$1.messageReceived(AbstractIoFilterChain.java:800)
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
> > > >>
> >
org.red5.io.filter.ExecutorFilter.processEvent
> (ExecutorFilter.java:
> > > >> 355)
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
> > > >>
>
org.red5.io.filter.ExecutorFilter$ProcessEventsRunnable.run
> > > >>
(ExecutorFilter.java:416)
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
> > > >>
> >
java.util.concurrent.ThreadPoolExecutor$Worker.runTask
(Unknown
> > > Source)
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
> > > >>
> >
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
> > > >>
java.lang.Thread.run(Unknown Source)
> > > >>
> > > >>
> > > >>
> > > >> and when i try and stop
the windows service I get
> errors
> > like these
> > > >>
> > > >> INFO | jvm 3 |
2007/04/28 20:30:25 |
> > "pool-2-thread-99" prio=6
> > > >> tid=0x6e6fe400 nid=0xe3c
waiting on condition
> > > [0x7077f000..0x7077fa98]
> > > >> INFO | jvm 3 |
2007/04/28 20:30:25
> > > | java.lang.Thread.State :
> > > >> WAITING (parking)
> > > >> INFO | jvm 3 |
2007/04/28 20:30:25 | at
> > > >>
sun.misc.Unsafe.park(Native Method)
> > > >> INFO | jvm 3 |
2007/04/28 20:30:25 | -
> parking
> > to wait for
> > > >> <0x09de5318> (a
> > >
java.util.concurrent.SynchronousQueue$TransferStack)
> > > >> INFO | jvm 3 |
2007/04/28 20:30:25 | at
> > > >>
java.util.concurrent.locks.LockSupport.park
> (Unknown Source)
> > > >> INFO | jvm 3 |
2007/04/28 20:30:25 | at
> > > >>
> >
>
java.util.concurrent.SynchronousQueue$TransferStack.awaitFul
fill
> > > >> (Unknown
> > > >> Source)
> > > >> INFO | jvm 3 |
2007/04/28 20:30:25 | at
> > > >>
>
java.util.concurrent.SynchronousQueue$TransferStack.transfer
> > > >> (Unknown Source)
> > > >> INFO | jvm 3 |
2007/04/28 20:30:25 | at
> > > >>
java.util.concurrent.SynchronousQueue.take(Unknown
> Source)
> > > >> INFO | jvm 3 |
2007/04/28 20:30:25 | at
> > > >>
java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
> > Source)
> > > >> INFO | jvm 3 |
2007/04/28 20:30:25 | at
> > > >>
> >
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
> > > >>
> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 |
> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | JNI global
> > > references: 640
> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 |
> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | Heap
> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | def new
> generation
> > > total
> > > >> 72768K, used 49874K
[0x02690000, 0x07580000,
> 0x09cb0000)
> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | eden space
> > 64704K, 65%
> > > >> used
> > > >> [0x02690000, 0x04ff8020,
0x065c0000)
> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | from space
> > 8064K, 92%
> > > >> used
> > > >> [0x06da0000, 0x074ec890,
0x07580000)
> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | to space
> > 8064K, 0%
> > > >> used
> > > >> [0x065c0000, 0x065c0000,
0x06da0000)
> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | tenured
> > generation
> > > total
> > > >> 967936K, used 29501K
[0x09cb0000, 0x44df0000,
> 0x62690000)
> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | the space
> > > 967936K, 3%
> > > >> used [0x09cb0000,
0x0b97f798, 0x0b97f800, 0x44df0000)
> > > >> INFO | jvm 3 |
2007/04/28 20:30:26
> | compacting perm
> > > gen total
> > > >> 12800K, used 12747K
[0x62690000, 0x63310000,
> 0x66690000)
> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | the space
> > > 12800K, 99%
> > > >> used
> > > >> [0x62690000, 0x63302f20,
0x63303000, 0x63310000)
> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | No shared
> spaces
> > > configured.
> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 |
> > > >> ERROR | wrapper |
2007/04/28 20:30:29 | JVM did
> not exit on
> > > request,
> > > >> terminated
> > > >>
> > > >>
> > > >> and another server gave
us this but only when trying to
> > stop the
> > > >> service
> > > >>
> > > >> INFO | jvm 1 |
2007/04/28 20:38:50 | Server daemon
> > died!
> > > >> INFO | jvm 1 |
2007/04/28 20:38:50 |
> > > java.lang.OutOfMemoryError :
> > > >> unable to create new
native thread
> > > >> STATUS | wrapper |
2007/04/28 20:38:50 | Filter
> trigger
> > matched.
> > > >> Restarting JVM.
> > > >> INFO | jvm 1 |
2007/04/28 20:38:50 | at
> > > >>
java.lang.Thread.start0(Native Method)
> > > >> INFO | jvm 1 |
2007/04/28 20:38:50 | at
> > > >>
java.lang.Thread.start(Unknown Source)
> > > >> INFO | jvm 1 |
2007/04/28 20:38:50 | at
> > > >>
>
org.tanukisoftware.wrapper.WrapperManager.privilegedStopInne
r
> > > >> (
WrapperManager.java:3152)
> > > >> INFO | jvm 1 |
2007/04/28 20:38:50 | at
> > > >>
org.tanukisoftware.wrapper.WrapperManager.handleSocket
> > > >>
(WrapperManager.java:3797)
> > > >> INFO | jvm 1 |
2007/04/28 20:38:50 | at
> > > >>
org.tanukisoftware.wrapper.WrapperManager.run
> > ( WrapperManager.java:
> > > >> 4084)
> > > >> INFO | jvm 1 |
2007/04/28 20:38:50 | at
> > > >>
java.lang.Thread.run(Unknown Source)
> > > >> ERROR | wrapper |
2007/04/28 20:39:27 | Shutdown
> failed:
> > > Timed out
> > > >> waiting for signal from
JVM.
> > > >> STATUS | wrapper |
2007/04/28 20:39:27 | Dumping
> JVM state.
> > > >> ERROR | wrapper |
2007/04/28 20:39:31 | JVM did
> not exit on
> > > request,
> > > >> terminated
> > > >>
> > > >>
_______________________________________________
> > > >> Red5 mailing list
> > > >> Red5osflash.org <mailto:Red5osflash.org>
> <mailto:Red5osflash.org
<mailto:Red5osflash.org>>
> > <mailto:Red5osflash.org
<mailto:Red5osflash.org>
> <mailto:Red5osflash.org
<mailto:Red5osflash.org>>>
> > > >>
>
http://osflash.org/mailman/listinfo/red5_osflash.org
> <http://osflash.org/mailman/listinfo/red5_osflash.org>
> > > >>
> > > >
> > > >
> > > >
_______________________________________________
> > > > Red5 mailing list
> > > > Red5osflash.org
<mailto:Red5osflash.org>
> <mailto:Red5osflash.org
<mailto:Red5osflash.org>>
> > <mailto:Red5osflash.org
<mailto:Red5osflash.org> <mailto:
> Red5osflash.org <mailto:Red5osflash.org>>>
> > > >
http://osflash.org/mailman/listinfo/red5_osflash.org
> > < http://osflash.org/mailman/listinfo/red5_osflash.org>
> > > >
> > > >
> > >
> > >
> > >
_______________________________________________
> > > Red5 mailing list
> > > Red5osflash.org
<mailto:Red5osflash.org>
> <mailto:Red5osflash.org
<mailto:Red5osflash.org>> <mailto:
> > Red5osflash.org <mailto:Red5osflash.org>
> <mailto:Red5osflash.org
<mailto:Red5osflash.org>>>
> > >
http://osflash.org/mailman/listinfo/red5_osflash.org
> > >
> > >
> > >
> > >
> > > --
> > > I cannot tell why this heart
languishes in silence. It is
> for small
> > > needs it never asks, or knows or
remembers. -- Tagore
> > >
> > > Best Regards
> > > Steven Gong
> > >
> >
>
------------------------------------------------------------
------------
> >
> > >
> > >
_______________________________________________
> > > Red5 mailing list
> > > Red5osflash.org
<mailto:Red5osflash.org>
> <mailto:Red5osflash.org
<mailto:Red5osflash.org>>
> > >
http://osflash.org/mailman/listinfo/red5_osflash.org
> > >
> >
> >
> >
_______________________________________________
> > Red5 mailing list
> > Red5osflash.org <mailto:Red5osflash.org>
> <mailto:Red5osflash.org
<mailto:Red5osflash.org>>
> >
http://osflash.org/mailman/listinfo/red5_osflash.org
> >
> >
> >
> >
> > --
> > I cannot tell why this heart languishes in
silence. It is for small
> > needs it never asks, or knows or remembers.
-- Tagore
> >
> > Best Regards
> > Steven Gong
> >
>
------------------------------------------------------------
------------
> >
> >
_______________________________________________
> > Red5 mailing list
> > Red5osflash.org <mailto:Red5osflash.org>
> >
http://osflash.org/mailman/listinfo/red5_osflash.org
> >
>
>
> _______________________________________________
> Red5 mailing list
> Red5osflash.org <mailto:Red5osflash.org>
>
http://osflash.org/mailman/listinfo/red5_osflash.org
>
>
>
>
> --
> I cannot tell why this heart languishes in silence. It
is for small
> needs it never asks, or knows or remembers. -- Tagore
>
> Best Regards
> Steven Gong
>
------------------------------------------------------------
------------
>
> _______________________________________________
> Red5 mailing list
> Red5osflash.org
>
http://osflash.org/mailman/listinfo/red5_osflash.org
>
_______________________________________________
Red5 mailing list
Red5osflash.org
http://osflash.org/mailman/listinfo/red5_osflash.org
|
|
| Re: server hanging |
Australia |
2007-04-29 01:04:08 |
Ok the patch is in may take a while to work out if there is
problems.
Dan Rossi wrote:
> Ahh yep i get a tonne of these thrown
>
> if (!(message instanceof IStreamData)) {
> throw new RuntimeException(
> "expected IStreamData but
got " +
> message.getClass() + " (type " +
message.getDataType() + ")");
> }
>
> something up there.
>
>
> Steven Gong wrote:
>
>> Don't know if it helps. Try to add a line:
>> if (message == null) return false;
>> at the beginning of okayToSendMessage() in
PlaylistSubscriberStream.
>>
>> On 4/29/07, *Dan Rossi* <spamelectroteque.org
>> <mailto:spamelectroteque.org>>
wrote:
>>
>> OK, which file / line ?
>>
>> Steven Gong wrote:
>> > It's easy to make a workaround for this in
okayToSendMessage() to
>> > ignore the null pointer but I think we'd
better find the root cause
>> > this issue as this should be happen in
normal cases.
>> >
>> > On 4/29/07, *Dan Rossi* <spamelectroteque.org
>> <mailto:spamelectroteque.org>
>> > <mailto:spamelectroteque.org
<mailto:spamelectroteque.org>>>
>> wrote:
>> >
>> > OK it did it again, hanged without
warnings at all this time
>> until i
>> > tried to stop the service and then
displayed JVM dumps,
>> logged via the
>> > java service wrapper not red5. Ive put
red5 logging into
>> fatal to
>> > see if
>> > that helps ?
>> >
>> > Steven Gong wrote:
>> > > Will this bug hang the entire
server? I guess this should be a
>> > > contention that the
pengingMessage has been released
>> before it's
>> > > pushed to the consumer. So the
body of pendingMessage is
>> null and
>> > > causes that exception.
>> > >
>> > > Dan, if possible, please record
this as a tracker so that
>> this won't
>> > > be forgotten. Thanks.
>> > >
>> > > On 4/29/07, *Dan Rossi* <
spamelectroteque.org
>> <mailto:spamelectroteque.org>
>> > <mailto:spamelectroteque.org
<mailto:spamelectroteque.org>>
>> > > <mailto: spamelectroteque.org
>> <mailto:spamelectroteque.org>
<mailto:spamelectroteque.org
>> <mailto:spamelectroteque.org>>>>
>> > wrote:
>> > >
>> > > I already have for the
exception errors should i
>> duplicate the
>> > > report ?
>> > >
>> > > Thijs Triemstra | Collab
wrote:
>> > > > Please create a ticket
on JIRA..
>> > > >
>> > > > cheers,
>> > > >
>> > > > Thijs
>> > > >
>> > > > On Apr 29, 2007, at 2:43
AM, Dan Rossi wrote:
>> > > >
>> > > >
>> > > >> Hi this is a bizarre
one but after quite a while
>> having
>> > the server
>> > > >> up it
>> > > >> will hang without
any outofmemory error blocking
>> > connections it
>> > > could
>> > > >> because of these
kinds of exceptions
>> > > >>
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | [ERROR]
>> 745875
>> > > >> pool-3-thread-2:
>> > > >> (
>>
org.red5.server.stream.PlaylistSubscriberStream.error
>> > > >> ) Error while
pulling message.
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 |
>> > > >>
java.lang.NullPointerException
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
org.red5.server.stream.PlaylistSubscriberStream
>> > > >>
>> >
>>
$PlayEngine.okayToSendMessage(PlaylistSubscriberStream.java:
1241)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
org.red5.server.stream.PlaylistSubscriberStream
>> > > >>
>>
$PlayEngine.pullAndPush(PlaylistSubscriberStream.java:1301)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
>> >
>>
org.red5.server.stream.PlaylistSubscriberStream$PlayEngine.a
ccess
>> > > >>
$100(PlaylistSubscriberStream.java:682)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
org.red5.server.stream.PlaylistSubscriberStream.written
>> > > >>
(PlaylistSubscriberStream.java :535)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
org.red5.server.net.rtmp.BaseRTMPHandler.messageSent
>> > > >>
(BaseRTMPHandler.java:205)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
org.red5.server.net.rtmp.RTMPMinaIoHandler.messageSent
>> > > >>
(RTMPMinaIoHandler.java:169)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
org.apache.mina.common.support.AbstractIoFilterChain
>> > > >>
$TailFilter.messageSent(
>> AbstractIoFilterChain.java:716)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
>> > >
>> >
>>
org.apache.mina.common.support.AbstractIoFilterChain.callNex
tMessageSe
>> > > >> nt(
AbstractIoFilterChain.java:391)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
>> >
org.apache.mina.common.support.AbstractIoFilterChain.access$
1300
>> > > >>
(AbstractIoFilterChain.java :54)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
>> >
org.apache.mina.common.support.AbstractIoFilterChain$EntryIm
pl
>> > > >>
$1.messageSent(AbstractIoFilterChain.java :806)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
org.red5.io.filter.ExecutorFilter.processEvent
>> > > (ExecutorFilter.java:
>> > > >> 358)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
>>
org.red5.io.filter.ExecutorFilter$ProcessEventsRunnable.run
>> > > >>
(ExecutorFilter.java:416)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
java.util.concurrent.ThreadPoolExecutor$Worker.runTask
>> > (Unknown
>> > > Source)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
>> >
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
>> Source)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
java.lang.Thread.run(Unknown Source)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | [ERROR]
>> 745875
>> > > >> pool-3-thread-2:
>> > > >> (
org.red5.server.stream.PlaylistSubscriberStream.error
>> > > >> ) Error while
pulling message.
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 |
>> > > java.lang.RuntimeException:
>> > > >> expected IStreamData
but got class
>> > > >>
org.red5.server.net.rtmp.event.Unknown (type 91)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
org.red5.server.stream.PlaylistSubscriberStream
>> > > >>
$PlayEngine.okayToSendMessage(
>> > PlaylistSubscriberStream.java:1203)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
org.red5.server.stream.PlaylistSubscriberStream
>> > > >>
>>
$PlayEngine.pullAndPush(PlaylistSubscriberStream.java
:1301)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
>> >
>>
org.red5.server.stream.PlaylistSubscriberStream$PlayEngine.a
ccess
>> > > >>
$100(PlaylistSubscriberStream.java:682)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
>>
org.red5.server.stream.PlaylistSubscriberStream.written
>> > > >>
(PlaylistSubscriberStream.java:535)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
org.red5.server.net.rtmp.BaseRTMPHandler.messageSent
>> > > >>
(BaseRTMPHandler.java:205)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
org.red5.server.net.rtmp.RTMPMinaIoHandler.messageSent
>> > > >>
(RTMPMinaIoHandler.java:169)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
org.apache.mina.common.support.AbstractIoFilterChain
>> > > >>
$TailFilter.messageSent(
>> AbstractIoFilterChain.java:716)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
>> > >
>> >
>>
org.apache.mina.common.support.AbstractIoFilterChain.callNex
tMessageSe
>> > >
>> > > >> nt(
AbstractIoFilterChain.java:391)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
>> >
org.apache.mina.common.support.AbstractIoFilterChain.access$
1300
>> > > >>
(AbstractIoFilterChain.java :54)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
>> >
org.apache.mina.common.support.AbstractIoFilterChain$EntryIm
pl
>> > > >>
$1.messageSent(AbstractIoFilterChain.java :806)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
>> >
>>
org.red5.io.filter.ExecutorFilter.processEvent(ExecutorFilte
r.java:
>> > > >> 358)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
>>
org.red5.io.filter.ExecutorFilter$ProcessEventsRunnable.run
>> > > >>
(ExecutorFilter.java:416)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
java.util.concurrent.ThreadPoolExecutor$Worker.runTask
>> > (Unknown
>> > > Source)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
>> >
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
>> Source)
>> > > >> INFO | jvm 1 |
2007/04/28 13:56:36 | at
>> > > >>
java.lang.Thread.run(Unknown Source)
>> > > >> INFO | jvm 1 |
2007/04/28 14:01:52 | [ERROR]
>> 1062437
>> > > >>
DefaultQuartzScheduler_Worker-8:(
>> > > >>
>>
org.red5.server.scheduling.QuartzSchedulingService.error
>> > ) Job
>> > > >>
>>
org.red5.server.net.rtmp.RTMPConnection$KeepAliveJobd0570e
>> > > execution
>> > > >> failed: null
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | [ERROR]
>> 1083125
>> > > >> pool-3-thread-6:(
>> >
org.red5.server.service.ServiceInvoker.error
>> > > ) Error
>> > > >> executing call:
Service: null Method: closeStream Num
>> > Params: 0
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | [ERROR]
>> 1083141
>> > > >> pool-3-thread-6:(
>> >
org.red5.server.service.ServiceInvoker.error )
>> > > >> Service
>> > > >> invocation error
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 |
>> > > >>
java.lang.reflect.InvocationTargetException
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
>> > > >>
>>
sun.reflect.GeneratedMethodAccessor21.invoke(Unknown
Source)
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
>> > > >>
sun.reflect.DelegatingMethodAccessorImpl.invoke
>> (Unknown
>> > Source)
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
>> > > >>
java.lang.reflect.Method.invoke(Unknown Source)
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
>> > > >>
>> > >
>> >
>>
org.red5.server.service.ServiceInvoker.invoke(ServiceInvoker
.java:181)
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
>> > > >>
>> > >
>> >
>>
org.red5.server.net.rtmp.RTMPHandler.invokeCall(RTMPHandler.
java :174)
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
>> > > >>
org.red5.server.net.rtmp.RTMPHandler.onInvoke
>> > (RTMPHandler.java:355)
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
>> > > >>
>>
org.red5.server.net.rtmp.BaseRTMPHandler.messageReceived
>> > > >> (
BaseRTMPHandler.java:126)
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
>> > > >>
>>
org.red5.server.net.rtmp.RTMPMinaIoHandler.messageReceived
>> > > >> (
RTMPMinaIoHandler.java:116)
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
>> > > >>
org.apache.mina.common.support.AbstractIoFilterChain
>> > > >>
$TailFilter.messageReceived(
>> AbstractIoFilterChain.java:703)
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
>> > > >>
>> > >
>> >
>>
org.apache.mina.common.support.AbstractIoFilterChain.callNex
tMessageRe
>>
>> > >
>> > > >>
ceived(AbstractIoFilterChain.java:362)
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
>> > > >>
>> >
>>
org.apache.mina.common.support.AbstractIoFilterChain.access$
1200
>> > > >>
(AbstractIoFilterChain.java :54)
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
>> > > >>
>> >
org.apache.mina.common.support.AbstractIoFilterChain$EntryIm
pl
>> > > >>
$1.messageReceived(AbstractIoFilterChain.java:800)
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
>> > > >>
>> >
org.red5.io.filter.ExecutorFilter.processEvent
>> (ExecutorFilter.java:
>> > > >> 355)
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
>> > > >>
>>
org.red5.io.filter.ExecutorFilter$ProcessEventsRunnable.run
>> > > >>
(ExecutorFilter.java:416)
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
>> > > >>
>> >
java.util.concurrent.ThreadPoolExecutor$Worker.runTask
(Unknown
>> > > Source)
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
>> > > >>
>> >
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
>> Source)
>> > > >> INFO | jvm 1 |
2007/04/28 14:02:13 | at
>> > > >>
java.lang.Thread.run(Unknown Source)
>> > > >>
>> > > >>
>> > > >>
>> > > >> and when i try and
stop the windows service I get
>> errors
>> > like these
>> > > >>
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:25 |
>> > "pool-2-thread-99" prio=6
>> > > >> tid=0x6e6fe400
nid=0xe3c waiting on condition
>> > > [0x7077f000..0x7077fa98]
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:25
>> > > | java.lang.Thread.State
:
>> > > >> WAITING (parking)
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:25 | at
>> > > >>
sun.misc.Unsafe.park(Native Method)
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:25 | -
>> parking
>> > to wait for
>> > > >> <0x09de5318>
(a
>> > >
java.util.concurrent.SynchronousQueue$TransferStack)
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:25 | at
>> > > >>
java.util.concurrent.locks.LockSupport.park
>> (Unknown Source)
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:25 | at
>> > > >>
>> >
>>
java.util.concurrent.SynchronousQueue$TransferStack.awaitFul
fill
>> > > >> (Unknown
>> > > >> Source)
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:25 | at
>> > > >>
>>
java.util.concurrent.SynchronousQueue$TransferStack.transfer
>> > > >> (Unknown Source)
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:25 | at
>> > > >>
java.util.concurrent.SynchronousQueue.take(Unknown
>> Source)
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:25 | at
>> > > >>
java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
>> > Source)
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:25 | at
>> > > >>
>> >
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
>> Source)
>> > > >>
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 |
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | JNI global
>> > > references: 640
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 |
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | Heap
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | def new
>> generation
>> > > total
>> > > >> 72768K, used 49874K
[0x02690000, 0x07580000,
>> 0x09cb0000)
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | eden space
>> > 64704K, 65%
>> > > >> used
>> > > >> [0x02690000,
0x04ff8020, 0x065c0000)
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | from space
>> > 8064K, 92%
>> > > >> used
>> > > >> [0x06da0000,
0x074ec890, 0x07580000)
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | to space
>> > 8064K, 0%
>> > > >> used
>> > > >> [0x065c0000,
0x065c0000, 0x06da0000)
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | tenured
>> > generation
>> > > total
>> > > >> 967936K, used 29501K
[0x09cb0000, 0x44df0000,
>> 0x62690000)
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | the space
>> > > 967936K, 3%
>> > > >> used [0x09cb0000,
0x0b97f798, 0x0b97f800, 0x44df0000)
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:26
>> | compacting perm
>> > > gen total
>> > > >> 12800K, used 12747K
[0x62690000, 0x63310000,
>> 0x66690000)
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | the space
>> > > 12800K, 99%
>> > > >> used
>> > > >> [0x62690000,
0x63302f20, 0x63303000, 0x63310000)
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 | No shared
>> spaces
>> > > configured.
>> > > >> INFO | jvm 3 |
2007/04/28 20:30:26 |
>> > > >> ERROR | wrapper |
2007/04/28 20:30:29 | JVM did
>> not exit on
>> > > request,
>> > > >> terminated
>> > > >>
>> > > >>
>> > > >> and another server
gave us this but only when trying to
>> > stop the
>> > > >> service
>> > > >>
>> > > >> INFO | jvm 1 |
2007/04/28 20:38:50 | Server daemon
>> > died!
>> > > >> INFO | jvm 1 |
2007/04/28 20:38:50 |
>> > > java.lang.OutOfMemoryError :
>> > > >> unable to create new
native thread
>> > > >> STATUS | wrapper |
2007/04/28 20:38:50 | Filter
>> trigger
>> > matched.
>> > > >> Restarting JVM.
>> > > >> INFO | jvm 1 |
2007/04/28 20:38:50 | at
>> > > >>
java.lang.Thread.start0(Native Method)
>> > > >> INFO | jvm 1 |
2007/04/28 20:38:50 | at
>> > > >>
java.lang.Thread.start(Unknown Source)
>> > > >> INFO | jvm 1 |
2007/04/28 20:38:50 | at
>> > > >>
>>
org.tanukisoftware.wrapper.WrapperManager.privilegedStopInne
r
>> > > >> (
WrapperManager.java:3152)
>> > > >> INFO | jvm 1 |
2007/04/28 20:38:50 | at
>> > > >>
org.tanukisoftware.wrapper.WrapperManager.handleSocket
>> > > >>
(WrapperManager.java:3797)
>> > > >> INFO | jvm 1 |
2007/04/28 20:38:50 | at
>> > > >>
org.tanukisoftware.wrapper.WrapperManager.run
>> > ( WrapperManager.java:
>> > > >> 4084)
>> > > >> INFO | jvm 1 |
2007/04/28 20:38:50 | at
>> > > >>
java.lang.Thread.run(Unknown Source)
>> > > >> ERROR | wrapper |
2007/04/28 20:39:27 | Shutdown
>> failed:
>> > > Timed out
>> > > >> waiting for signal
from JVM.
>> > > >> STATUS | wrapper |
2007/04/28 20:39:27 | Dumping
>> JVM state.
>> > > >> ERROR | wrapper |
2007/04/28 20:39:31 | JVM did
>> not exit on
>> > > request,
>> > > >> terminated
>> > > >>
>> > > >>
_______________________________________________
>> > > >> Red5 mailing list
>> > > >> Red5osflash.org <mailto:Red5osflash.org>
>> <mailto:Red5osflash.org
<mailto:Red5osflash.org>>
>> > <mailto:Red5osflash.org
<mailto:Red5osflash.org>
>> <mailto:Red5osflash.org
<mailto:Red5osflash.org>>>
>> > > >>
>>
http://osflash.org/mailman/listinfo/red5_osflash.org
>> <http://osflash.org/mailman/listinfo/red5_osflash.org>
>> > > >>
>> > > >
>> > > >
>> > > >
_______________________________________________
>> > > > Red5 mailing list
>> > > > Red5osflash.org <mailto:Red5osflash.org>
>> <mailto:Red5osflash.org
<mailto:Red5osflash.org>>
>> > <mailto:Red5osflash.org
<mailto:Red5osflash.org> <mailto:
>> Red5osflash.org <mailto:Red5osflash.org>>>
>> > > >
http://osflash.org/mailman/listinfo/red5_osflash.org
>> > < http://osflash.org/mailman/listinfo/red5_osflash.org>
>> > > >
>> > > >
>> > >
>> > >
>> > >
_______________________________________________
>> > > Red5 mailing list
>> > > Red5osflash.org
<mailto:Red5osflash.org>
>> <mailto:Red5osflash.org
<mailto:Red5osflash.org>> <mailto:
>> > Red5osflash.org
<mailto:Red5osflash.org>
>> <mailto:Red5osflash.org
<mailto:Red5osflash.org>>>
>> > >
http://osflash.org/mailman/listinfo/red5_osflash.org
>> > >
>> > >
>> > >
>> > >
>> > > --
>> > > I cannot tell why this heart
languishes in silence. It is
>> for small
>> > > needs it never asks, or knows or
remembers. -- Tagore
>> > >
>> > > Best Regards
>> > > Steven Gong
>> > >
>> >
>>
------------------------------------------------------------
------------
>> >
>> > >
>> > >
_______________________________________________
>> > > Red5 mailing list
>> > > Red5osflash.org
<mailto:Red5osflash.org>
>> <mailto:Red5osflash.org
<mailto:Red5osflash.org>>
>> > >
http://osflash.org/mailman/listinfo/red5_osflash.org
>> > >
>> >
>> >
>> >
_______________________________________________
>> > Red5 mailing list
>> > Red5osflash.org
<mailto:Red5osflash.org>
>> <mailto:Red5osflash.org
<mailto:Red5osflash.org>>
>> >
http://osflash.org/mailman/listinfo/red5_osflash.org
>> >
>> >
>> >
>> >
>> > --
>> > I cannot tell why this heart languishes in
silence. It is for small
>> > needs it never asks, or knows or
remembers. -- Tagore
>> >
>> > Best Regards
>> > Steven Gong
>> >
>>
------------------------------------------------------------
------------
>> >
>> >
_______________________________________________
>> > Red5 mailing list
>> > Red5osflash.org <mailto:Red5osflash.org>
>> >
http://osflash.org/mailman/listinfo/red5_osflash.org
>> >
>>
>>
>>
_______________________________________________
>> Red5 mailing list
>> Red5osflash.org <mailto:Red5osflash.org>
>>
http://osflash.org/mailman/listinfo/red5_osflash.org
>>
>>
>>
>>
>> --
>> I cannot tell why this heart languishes in silence.
It is for small
>> needs it never asks, or knows or remembers. --
Tagore
>>
>> Best Regards
>> Steven Gong
>>
------------------------------------------------------------
------------
>>
>> _______________________________________________
>> Red5 mailing list
>> Red5osflash.org
>>
http://osflash.org/mailman/listinfo/red5_osflash.org
>>
>>
>
>
> _______________________________________________
> Red5 mailing list
> Red5osflash.org
>
http://osflash.org/mailman/listinfo/red5_osflash.org
>
>
_______________________________________________
Red5 mailing list
Red5osflash.org
http://osflash.org/mailman/listinfo/red5_osflash.org
|
|
[1-2]
|
|