Dan Rossi wrote:
> Some interesting info from the logs, here is at the
point of rebooting,
> would the logs be causing memory leaks ?
>
> [INFO] 7299657 pool-3-thread-16:(
VODApplication.appJoin ) Client joined
> app 1038 at time 1177748584250
> [INFO] 7844 pool-3-thread-9:( VODApplication.appJoin )
Client joined app
> 0 at time 1177748620265
>
Ok there is a pattern, server reboots, hangs or gets
outofmemory at
client number 1038, that doesnt really explain concurrents
though.
> Dan Rossi wrote:
>
>> Looks like even when load balanced it blocks
connections after time. Its
>> all the exceptions im sure of it, no out of memory
errors which is
>> bizarre until i try and reboot the service which it
logs all the jvm
>> dumps because it hanged, ive kept the threadcount
to 4 as setting it to
>> the max it can do causes issues which was 2700.
>>
>> This one is fairly bizarre as it wasnt an
outofmemory error as i usually
>> get, we load files from a network drive using a UNC
name like so
>> file:////computername/sharename
>>
>> jvm 2 | 2007/04/28 00:45:32 |
java.io.IOException: The specified
>> network name is no longer available
>> INFO | jvm 2 | 2007/04/28 00:45:32 | at
>> sun.nio.ch.FileDispatcher.read0(Native Method)
>> INFO | jvm 2 | 2007/04/28 00:45:32 | at
>> sun.nio.ch.FileDispatcher.read(Unknown Source)
>> INFO | jvm 2 | 2007/04/28 00:45:32 | at
>> sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown
Source)
>> INFO | jvm 2 | 2007/04/28 00:45:32 | at
>> sun.nio.ch.IOUtil.read(Unknown Source)
>> INFO | jvm 2 | 2007/04/28 00:45:32 | at
>> sun.nio.ch.FileChannelImpl.read(Unknown Source)
>> INFO | jvm 2 | 2007/04/28 00:45:32 | at
>>
org.red5.io.flv.impl.FLVReader.fillBuffer(FLVReader.java:319
)
>> INFO | jvm 2 | 2007/04/28 00:45:32 | at
>>
org.red5.io.flv.impl.FLVReader.fillBuffer(FLVReader.java:268
)
>> INFO | jvm 2 | 2007/04/28 00:45:32 | at
>>
org.red5.io.flv.impl.FLVReader.readTag(FLVReader.java:545)
>>
>> Dan Rossi wrote:
>>
>>
>>> Seemed to kind of stablise for a while, however
im looking at the
>>> performance monitor and the page faults column
is high, is this an issue
>>> ? It reached about 1900 threads and then
rebooted from the outofmemory
>>> errors. I set the threadcount to 2700 as that
is aparantly the max the
>>> JVM can do.
>>>
>>> Dan Rossi wrote:
>>>
>>>
>>>
>>>> I tried adding these into the wrapper
config to no avail, causes to use
>>>> more virtual memory , and memory gets quite
large
>>>>
>>>> wrapper.java.additional.1=-Xms1024m
>>>> wrapper.java.additional.2=-Xmx1024m
>>>>
#wrapper.java.additional.3=-XX:NewSize=512m
>>>>
#wrapper.java.additional.4=-XX:MaxNewSize=512m
>>>>
#wrapper.java.additional.5=-XX:SurvivorRatio=8
>>>>
>>>> Dan Rossi wrote:
>>>>
>>>>
>>>>
>>>>
>>>>> Also is there any reason why we need so
much memory resources, our
>>>>> traffic has always been the same, and
windows media only uses up about
>>>>> 500MB. will we need to get 4GB now ?
>>>>>
>>>>> Dan Rossi wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> It seems after messing with the
memory limits, red5 rejects connections
>>>>>> but doesnt throw exceptions or out
of memory errors, and cant connect
>>>>>> until it reboots. Is there any
recommendations of tweaking a windows
>>>>>> server with 2GB of ram, ie tweaking
the memory and cpu allocation and
>>>>>> pagefile size etc. what other java
options should i be adding ? How can
>>>>>> i monitor what red5 is doing
internally ?
>>>>>>
>>>>>>
>>>>>> Dan Rossi wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Nothing special, its being
caused every 2 hours.
>>>>>>>
>>>>>>> Main server is just running
red5, the 2 slave machines are also running
>>>>>>> windows media and IIS which
were also getting the errors however the
>>>>>>> server has 2GB of ram and the
resource monitor only shows 700MB being
>>>>>>> used etc. Its only started
happening since updates not long ago.
>>>>>>>
>>>>>>> Lenny Sorey wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Dan,
>>>>>>>>
>>>>>>>> Forgive me if you have
answered these questions previously.
>>>>>>>>
>>>>>>>> What are you running in
RED5 which causes the out of memory problem.
>>>>>>>> Video Streaming, Live
Broadcast a combination of both?
>>>>>>>>
>>>>>>>> What all do you have
actually running on these servers.
>>>>>>>>
>>>>>>>> RED5 (jetty), IIS, SQL
Server, etc?
>>>>>>>>
>>>>>>>> How many users are
connected when this error occurs?
>>>>>>>>
>>>>>>>> Can you tell in the scheme
of things when and where the out of memory
>>>>>>>> problems are occurring?
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Lenny
>>>>>>>>
>>>>>>>> How long does RED5 until
you receive your Out of Memory error message
>>>>>>>> in the logs.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 4/26/07, *Dan Rossi*
<spam electroteque.org
>>>>>>>> <mailto:spamelectroteque.org>> wrote:
>>>>>>>>
>>>>>>>> Hi we are constantly
getting these errors on our windows 2003 servers
>>>>>>>> after the latest svn
updates. Im trying to work out how to tweak the
>>>>>>>> resources on our
windows servers to fix these kind of errors
>>>>>>>> causing the
>>>>>>>> java wrapper service to
reboot. Im trying to work out what are the
>>>>>>>> optimal memory and cpu
allocation settings to use and java options for
>>>>>>>> resources, we have 2G
of ram, and dual P4 3G on all machines, even
>>>>>>>> after
>>>>>>>> load balancing its
still doing this. Has only just started happening.
>>>>>>>>
>>>>>>>> #
java.lang.OutOfMemoryError: requested 32756 bytes for
>>>>>>>> ChunkPool::allocate.
Out of swap space?
>>>>>>>>
>>>>>>>> Here is the advanced
log
>>>>>>>>
>>>>>>>> 0x48127800 JavaThread
"pool-2-thread-584" [_thread_blocked, id=8832]
>>>>>>>> 0x4741e400 JavaThread
"pool-2-thread-583" [_thread_blocked, id=2692]
>>>>>>>> 0x51bc4400 JavaThread
"pool-2-thread-582" [_thread_blocked, id=3668]
>>>>>>>> 0x5325f400 JavaThread
"pool-2-thread-581" [_thread_blocked, id=8752]
>>>>>>>> 0x47ba8800 JavaThread
"pool-2-thread-580" [_thread_blocked, id=9936]
>>>>>>>> 0x50fac400 JavaThread
"pool-2-thread-579" [_thread_blocked, id=10100]
>>>>>>>> 0x47ec1c00 JavaThread
"pool-2-thread-578" [_thread_blocked, id=7584]
>>>>>>>>
>>>>>>>>
>>>>>>>> VM Arguments:
>>>>>>>> jvm_args: -Xms512m
-Xmx1000m -Djava.library.path=../lib
>>>>>>>>
-Dwrapper.key=z6ob3QwdVVUuY_JG -Dwrapper.port=32000
>>>>>>>>
-Dwrapper.use_system_time=TRUE -Dwrapper.version=3.1.2
>>>>>>>>
-Dwrapper.native_library=wrapper -Dwrapper.service=TRUE
>>>>>>>>
-Dwrapper.cpu.timeout=10 -Dwrapper.jvmid=1
>>>>>>>> java_command:
org.tanukisoftware.wrapper.WrapperSimpleApp
>>>>>>>>
org.red5.server.Standalone
>>>>>>>> Launcher Type:
SUN_STANDARD
>>>>>>>>
>>>>>>>> --------------- S Y S
T E M ---------------
>>>>>>>>
>>>>>>>> OS: Windows Server 2003
family Build 3790 Service Pack 1
>>>>>>>>
>>>>>>>> CPU:total 2 family 15,
cmov, cx8, fxsr, mmx, sse, sse2, ht
>>>>>>>>
>>>>>>>> Memory: 4k page,
physical 2030356k(997988k free), swap
>>>>>>>> 3359568k(2099096k
>>>>>>>> free)
>>>>>>>>
>>>>>>>> vm_info: Java
HotSpot(TM) Client VM (1.6.0_01-b06) for windows-x86,
>>>>>>>> built on Mar 14 2007
00:24:02 by "java_re" with unknown MS VC++:1310
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
_______________________________________________
>>>>>>>> Red5 mailing list
>>>>>>>> Red5osflash.org <mailto:Red5osflash.org>
>>>>>>>>
http://osflash.org/mailman/listinfo/red5_osflash.org
>>>>>>>> <http://osflash.org/mailman/listinfo/red5_osflash.org>
>>>>>>>>
>>>>>>>>
>>>>>>>>
------------------------------------------------------------
------------
>>>>>>>>
>>>>>>>>
_______________________________________________
>>>>>>>> Red5 mailing list
>>>>>>>> Red5osflash.org
>>>>>>>>
http://osflash.org/mailman/listinfo/red5_osflash.org
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
_______________________________________________
>>>>>>> Red5 mailing list
>>>>>>> Red5osflash.org
>>>>>>>
http://osflash.org/mailman/listinfo/red5_osflash.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
_______________________________________________
>>>>>> Red5 mailing list
>>>>>> Red5osflash.org
>>>>>>
http://osflash.org/mailman/listinfo/red5_osflash.org
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
_______________________________________________
>>>>> Red5 mailing list
>>>>> Red5osflash.org
>>>>>
http://osflash.org/mailman/listinfo/red5_osflash.org
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
_______________________________________________
>>>> Red5 mailing list
>>>> Red5osflash.org
>>>>
http://osflash.org/mailman/listinfo/red5_osflash.org
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
_______________________________________________
>>> Red5 mailing list
>>> Red5osflash.org
>>>
http://osflash.org/mailman/listinfo/red5_osflash.org
>>>
>>>
>>>
>>>
>> _______________________________________________
>> Red5 mailing list
>> Red5osflash.org
>>
http://osflash.org/mailman/listinfo/red5_osflash.org
>>
>>
>>
>
>
> _______________________________________________
> Red5 mailing list
> Red5osflash.org
>
http://osflash.org/mailman/listinfo/red5_osflash.org
>
>
_______________________________________________
Red5 mailing list
Red5osflash.org
http://osflash.org/mailman/listinfo/red5_osflash.org
|
