Handling of video images

Hello all!

Is there already a plan on how to pass the video image of a
video SIP call
from the CallSessionImpl to the GUI front-end? I am
currently working on
embedding the SIP communicator into our application and need
to integrate
the video component. If not, I could propose a way and start
implementing it
if it is accepted (at least the back-end side).

We at Rowa need an embeddable Java video conference libary
and I am working
on using SIP communicator as a basis. We'll release the
(modified) sources
in accordance with the LGPL of course, but since I had to
restructure it
pretty much I don't know if any of it will be useful for
integration with
the main SIP Communicator. I hope that I can help out with
some things,
however (mainly robustness of the SIP backend).

Once I am through with embedding SIP Communicator, I'll post
a report of
what I have done.

Thanks to all contributors for all the great work you have
done so far.

Regards
Michael Koch

Hello Michael,

Right now the media service is simply popping out a frame
rendering the
video flow. I guess you've noticed this already. This of
course is only
temporary and we'll have to think of a more elaborate way to
handle it
in the future.

Possible solutions may include:

1. Adding a method to CallSession that allows it to return a
visual
component that the UI (or any other bundle) would be able to
display. We
would also have to add methods that would allow users to
register
listeners and get notified when such a component becomes
available.
(Pretty much the way things work with the JMF Player
interface)

2. Allowing other modules to register container components
where the
media service would be supposed render all visual content.

3. Do not pass components across modules, and instead,
define an API in
CallSession that would give you some control over its visual
components
(move, resize, dock).

The tricky part for 1 and 2 is how to handle this in a way
that is
independent from a particular UI lib. I don't really see a
way to do so
unless we settle for AWT components and hope that they will
be supported
by other UI libs (which is the case with swing and swt)

3 on the other hand, would give us the independence but
would add a lot
of complexity to the implementation of Call Sessions.

Personally, I am not sure which is the best way to go. Do
you have any
ideas?

Emil


Koch Michael wrote:
> Hello all!
> 
> Is there already a plan on how to pass the video image
of a video SIP call
> from the CallSessionImpl to the GUI front-end? I am
currently working on
> embedding the SIP communicator into our application and
need to integrate
> the video component. If not, I could propose a way and
start implementing it
> if it is accepted (at least the back-end side).
> 
> We at Rowa need an embeddable Java video conference
libary and I am working
> on using SIP communicator as a basis. We'll release the
(modified) sources
> in accordance with the LGPL of course, but since I had
to restructure it
> pretty much I don't know if any of it will be useful
for integration with
> the main SIP Communicator. I hope that I can help out
with some things,
> however (mainly robustness of the SIP backend).
> 
> Once I am through with embedding SIP Communicator, I'll
post a report of
> what I have done.
> 
> Thanks to all contributors for all the great work you
have done so far.
> 
> Regards
> Michael Koch

------------------------------------------------------------
---------
To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
For additional commands, e-mail: dev-helpsip-communicator.dev.java.net

hello everybody,
im not sure if anyone has every thought about anything like
that or maybe 
there already is some kind of solution for this. im thinking
about a quality 
of service implementation for the SC doing two different
things. the 
probably more easy part to write a custom jmf rtpconnector
to be able to 
define the tos field for the ip pakets. after looking only
vaguely over the 
rtpmanager initialization in SC i saw that you are using the
addtarget 
functions which will be a problem when initializing with a
custom 
rtpconnector.

else has anyone ever thought about some kind of buffering
the incoming data 
and then selecting which to send out. let me give you any
example. you are 
doing a file transfer and have a audio phone conversation at
the same time. 
both data is being put in two different buffers like this

|   file     |         |  audio             |
| tranfer |         | conversation    |
|           |          |                       |
--------           ---------------- 
        
             <--- selector
          

and then you have some kind of selecting mechanism to decide
which data to 
send out first. the easiest way for this is probably to
define for example 
audio as priority 1 and a file transfer as prio 2 and then
the selector 
chooses the buffer prio 1 for as long as there is data in
that buffer. when 
there is nothing in there is checks buffer prio 2 and so
on.

does anyone have any clue if this idea is actually
realizable to program 
into the SC. i mean in a one man job and with a time frame
of about 1 month. 
ill be thankful for any kind of help. where to start or
anything that could 
help

thanks alot for any help/time!

regards

kolja






------------------------------------------------------------
---------
To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
For additional commands, e-mail: dev-helpsip-communicator.dev.java.net

Hello Kolja,

(see inline)

Kolja Ehlers wrote:
> hello everybody,
> im not sure if anyone has every thought about anything
like that or maybe 
> there already is some kind of solution for this. im
thinking about a quality 
> of service implementation for the SC doing two
different things. the 
> probably more easy part to write a custom jmf
rtpconnector to be able to 
> define the tos field for the ip pakets. after looking
only vaguely over the 
> rtpmanager initialization in SC i saw that you are
using the addtarget 
> functions which will be a problem when initializing
with a custom 
> rtpconnector.

This is easily changed, and besides, we might have to change
it anyway
since I heard Su Bing (who is working on SRTP) is also
having problems
with the same thing.

> else has anyone ever thought about some kind of
buffering the incoming data 
> and then selecting which to send out. let me give you
any example. you are 
> doing a file transfer and have a audio phone
conversation at the same time. 
> both data is being put in two different buffers like
this
> 
> |   file     |         |  audio             |
> | tranfer |         | conversation    |
> |           |          |                       |
> --------           ---------------- 
>         
>              <--- selector
>           
> 
> and then you have some kind of selecting mechanism to
decide which data to 
> send out first. the easiest way for this is probably to
define for example 
> audio as priority 1 and a file transfer as prio 2 and
then the selector 
> chooses the buffer prio 1 for as long as there is data
in that buffer. when 
> there is nothing in there is checks buffer prio 2 and
so on.
> 
> does anyone have any clue if this idea is actually
realizable to program 
> into the SC. i mean in a one man job and with a time
frame of about 1 month. 
> ill be thankful for any kind of help. 

Depends on what your goal is. In case you are only working
on a research
project and all you want to have at the end is a
proof-of-concept demo,
then it might feasible.

However, in case you want to use this in real world
conditions then it
is probably not the best approach. You'd have to modify all
stacks that
support file transfer, and make sure that they use your QoS
queues, and
it would still not prevent other applications from occupying
all
available bandwidth.

For a more reliable implementation you'd probably have to do
this in the
OS kernel itself. I think there is a fair amount of related
art so you
might want to look around for existing implementations.

Hope this helps
Emil

> where to start or anything that could 
> help
> 
> thanks alot for any help/time!
> 
> regards
> 
> kolja
> 
> 
> 
> 
> 
> 
>
------------------------------------------------------------
---------
> To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
> For additional commands, e-mail: dev-helpsip-communicator.dev.java.net
> 
> 

------------------------------------------------------------
---------
To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
For additional commands, e-mail: dev-helpsip-communicator.dev.java.net

Hi Kolja,

I'm working on SRTP support for SC. Since we are going to
use our
customized RTPConnector to handle RTP traffic, we are
responsible for
managing the "connection" (stream targets, etc).
So, there are some
logic we should take care. But if you have only one target,
you can
simply discard the addTarget() call and set your target
session
address in your RTPConnector, e.g. constructor. If you will
have
multiple destinations, you can use a vector to store the
addresses and
send every packets to all the targets.

Best regards,
Su

On 7/16/07, Emil Ivov <emchoemcho.com> wrote:
> Hello Kolja,
>
> (see inline)
>
> Kolja Ehlers wrote:
> > hello everybody,
> > im not sure if anyone has every thought about
anything like that or maybe
> > there already is some kind of solution for this.
im thinking about a quality
> > of service implementation for the SC doing two
different things. the
> > probably more easy part to write a custom jmf
rtpconnector to be able to
> > define the tos field for the ip pakets. after
looking only vaguely over the
> > rtpmanager initialization in SC i saw that you are
using the addtarget
> > functions which will be a problem when
initializing with a custom
> > rtpconnector.
>
> This is easily changed, and besides, we might have to
change it anyway
> since I heard Su Bing (who is working on SRTP) is also
having problems
> with the same thing.
>
> > else has anyone ever thought about some kind of
buffering the incoming data
> > and then selecting which to send out. let me give
you any example. you are
> > doing a file transfer and have a audio phone
conversation at the same time.
> > both data is being put in two different buffers
like this
> >
> > |   file     |         |  audio             |
> > | tranfer |         | conversation    |
> > |           |          |                       |
> > --------           ----------------
> >         
> >              <--- selector
> >           
> >
> > and then you have some kind of selecting mechanism
to decide which data to
> > send out first. the easiest way for this is
probably to define for example
> > audio as priority 1 and a file transfer as prio 2
and then the selector
> > chooses the buffer prio 1 for as long as there is
data in that buffer. when
> > there is nothing in there is checks buffer prio 2
and so on.
> >
> > does anyone have any clue if this idea is actually
realizable to program
> > into the SC. i mean in a one man job and with a
time frame of about 1 month.
> > ill be thankful for any kind of help.
>
> Depends on what your goal is. In case you are only
working on a research
> project and all you want to have at the end is a
proof-of-concept demo,
> then it might feasible.
>
> However, in case you want to use this in real world
conditions then it
> is probably not the best approach. You'd have to modify
all stacks that
> support file transfer, and make sure that they use your
QoS queues, and
> it would still not prevent other applications from
occupying all
> available bandwidth.
>
> For a more reliable implementation you'd probably have
to do this in the
> OS kernel itself. I think there is a fair amount of
related art so you
> might want to look around for existing
implementations.
>
> Hope this helps
> Emil
>
> > where to start or anything that could
> > help
> >
> > thanks alot for any help/time!
> >
> > regards
> >
> > kolja
> >
> >
> >
> >
> >
> >
> >
------------------------------------------------------------
---------
> > To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
> > For additional commands, e-mail: dev-helpsip-communicator.dev.java.net
> >
> >
>
>
------------------------------------------------------------
---------
> To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
> For additional commands, e-mail: dev-helpsip-communicator.dev.java.net
>
>

------------------------------------------------------------
---------
To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
For additional commands, e-mail: dev-helpsip-communicator.dev.java.net

Hello Emil!

I don't know SIP communicator much, and especially haven't
looked at the GUI
part, so I'm not really qualified to say which would be the
best approach.
I'll try anyway 

I think that, from the GUI point of view, the video image
has to be
associated with the CallParticipant who is shown. This way,
when there is a
conference call or multiple simultaneous calls, the GUI can
show the name of
the caller. Since the incoming media is thus associated with
the
CallParticipant, a media listener which can be registered on
objects of that
class may be useful. It would make the implementation kind
of difficult,
since the component has to be passed from media to protocol,
and from there
to gui.

Your solution 1 (extending the CallSession interface) has
the drawback that
the interface is currently used for the low-level tasks like
handling the
SDP data, which the GUI usually doesn't care about. I don't
know if the GUI
currently uses the media bundle directly at all?

Solutions 2 and 3 add more GUI knowledge to the media
bundle. Of course I'm
not the one to make the decisions, but for me it doesn't
feel right. On the
other hand, audio rendering and control is also done in
media, so perhaps it
would not be too bad after all.

On UI lib independence, I think it is also a matter of what
the media
library supplies to the media bundle. The players of JMF and
FMJ both use
some java.awt.Component subclass (I think), so I guess that
is all SIP
communicator needs to support. Since java.awt.Component has
been part of
Java since 1.0, I guess all alternative GUI libraries will
support it in
some way.

I don't know what would be the best approach. One probably
has to think
about the GUI for video calls first, and then decide how the
needed
information is best passed from back-end to front-end. If
someone wants to
do this part, I could help out with the back-end
implementation. For my
short-term needs, I have extended the
MediaEvent/MediaListener mechanism
which was already there and pass the component around in the
event. I don't
know if this is such a good solution, because:
* I'm not sure if I am using the event for the purpose it
was intended for
* It hooks the GUI directly to the media bundle, which I
don't know if this
is supposed to be done
but it works for me 
(I could create a patch if there is interest).

Regards
Michael Koch

> Right now the media service is simply popping out a
frame 
> rendering the
> video flow. I guess you've noticed this already. This
of 
> course is only
> temporary and we'll have to think of a more elaborate
way to handle it
> in the future.
> 
> Possible solutions may include:
> 
> 1. Adding a method to CallSession that allows it to
return a visual
> component that the UI (or any other bundle) would be
able to 
> display. We
> would also have to add methods that would allow users
to register
> listeners and get notified when such a component
becomes available.
> (Pretty much the way things work with the JMF Player
interface)
> 
> 2. Allowing other modules to register container
components where the
> media service would be supposed render all visual
content.
> 
> 3. Do not pass components across modules, and instead,
define 
> an API in
> CallSession that would give you some control over its
visual 
> components
> (move, resize, dock).
> 
> The tricky part for 1 and 2 is how to handle this in a
way that is
> independent from a particular UI lib. I don't really
see a 
> way to do so
> unless we settle for AWT components and hope that they
will 
> be supported
> by other UI libs (which is the case with swing and
swt)
> 
> 3 on the other hand, would give us the independence but
would 
> add a lot
> of complexity to the implementation of Call Sessions.
> 
> Personally, I am not sure which is the best way to go.
Do you have any
> ideas?
> 
> Emil