All your problems, solved ;)

Jonathan Chayce Dickinson wrote:
> Hey People,
> 
> I was perusing over a couple of MSDN articles, when I
came across DIME
> (Direct Internet Message Encapsulation). Jeez, I
thought, that would fix
> all the file issues in Jabber (like SI). So I wrote a
draft, any takers?
> What do you all think? Didn't want to push this forward
until I got some
> feedback.

Please don't waste our time with fanciful notions like
this.

I say it is a fanciful notion because:

1. DIME was some Microsoft technology.

2. Which they submitted to the IETF in 2001 and never
updated.

3. Which they then abandoned in favor of "SOAP Message
Transmission
Optimization Mechanism" (MTOM).

4. Which in turn basically has nothing to do with XMPP or
anything we
would work on here.

And I don't know why my name is on the protoXEP you wrote
up, because I
had nothing to do with this.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

On Mon Aug 27 20:55:05 2007, Jonathan Chayce Dickinson
wrote:
> I was perusing over a couple of MSDN articles, when I
came across 
> DIME (Direct Internet Message Encapsulation). Jeez, I
thought, that 
> would fix all the file issues in Jabber (like SI). So I
wrote a 
> draft, any takers? What do you all think? Didn't want
to push this 
> forward until I got some feedback.

DIME seems to have been orphaned by its authors, in favour
of 
soap12-mtom, so I'm not convinced it'd be a good plan to go
this 
route whatever the merits of the general concept.

However, as a general note, encapsulating small amounts of
"binary" 
data in base64 is generally okay - you'll recover the
majority of the 
33% bloat through compression (which you hopefully have via
TLS, and 
may have via other means).

This isn't suitable for large amounts of data, though, since

compression doesn't fully recover the overhead.

Moreover, compression algorithms will become
"poisoned" through 
trying to compress different types of data - typically via
skewing 
the Huffman encoder and destroying the backreference
"dictionary". 
This is especially true if the data being transferred is
already 
compressed, in this case you can observe surprisingly high
increases 
in data transfer cost (ie, negative compression).

But sending bulk data through XMPP is a bad idea for a
number of 
other reasons, not least of which it's trashing the server,
and 
clogging your XMPP channel (thus reducing its response
time).

A better method is to negotiate a peer-to-peer session
suited to the 
kind of transfer you're doing, although I admit that's not
exactly a 
ground-breaking suggestion. 

Dave.
-- 
Dave Cridland - mailto:davecridland.net - xmpp:dwdjabber.org
  -
acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

I'm sorry. Didn't mean any harm. I'm not too hot on
bandwidth here, so I 
didn't come across MTOM.

I wasn't wasting your time, quote: "Didn't want to push
this forward 
until I got some feedback". I don't like wasting
people's time, hell, I 
love helping people out and solving problems: don't reply to
this and 
you will never hear of it again; all it takes is "nope,
not a good 
idea," not something as violent as "fanciful"
(ouch!).

Read the spec and you will see that it has a lot to do with
XMPP: Namely 
(quoted from my draft):

1. DIME will rectify the absence of true In-Band
Bytestreams, as it 
defines methods to send multiple payloads in a atomical
message. Each 
payload in the message can reference other payloads in the
message.

XMPP needs, in my opinion, a way to negotiate real in-band
bytestreams. 
Base64 seems like a cop-out to me, it really does.

2. Furthermore compression methods used by servers (which
tend to be 
fast instead of -small-effective) will, at best, return the
data to near 
it's original length. If the data is presented to these
algorithms in 
its original state they may be able to compress it even
further.

Not everyone has a T3 connection at home, some of us live in
South 
Africa, and well, bandwidth here is rather expensive: and so
is a static 
IP (yes, we differentiate between them here, we /actually/
have 
non-static ones too). When you have to weigh up the value of
1MB, you 
will know.

3. Ultimately this protocol will smooth out the issues
plaguing Jabber 
regarding file sharing.

This was a hot topic on JDev not too long ago! Demand,
supply. If the 
developers are demanding it, I ASSURE you, the users will be
fast 
following (faster than any cheetah metaphores ;)

4. Almost all the existing XEPs should be able to derive
some further 
benefit from this extension (especially Jingle and SI).

That is a bit fanciful and ambitious, I admit. Add XHTML to
that list, 
and well there you go, not even nearly all, but 3, which is
a start.

(Just coming in on that regard, SOAP has nothing to do with
XMPP, but 
neither did SASL (at the start), or XHTML... And there is
SOAP over XMPP 
now)

And, well, *both* are XML and *both* suffer from the fact
that you can't 
package binary data reliably and efficiently in an XML
document. MTOM 
wouldn't exist if it wasn't a real problem. It would have
ended with DIME.

A lot of protocols do it, hell, even your cellphone does
something 
similar (which is why you can call someone and surf over
GPRS/3G/HSDPA 
at the same time).

In regard to Dave's email, about the negative compression
etc. very 
true, didn't think of that. Nice, constructive criticism.
Dave: the very 
reason I thought of it was because a P2P connection isn't
always 
possible, as in my case (I don't need it, but the awareness
is there). I 
was hoping to use the server to host the
Peer-To-Host-To-Peer connection 
instead of some 3rd party website/ftp server.

I saw a gap, jumped, it seemed like a good idea, but it
obviously 
wasn't. Sorry about putting your name on it Peter: it was in
the 
template, I thought it belonged there. I'm sorry I put this
out there, I 
could have just as easily made the whole thing a
undocumented protocol 
on the project I am working on, which actually, ironically,
REALLY needs 
it: and probably will land up doing now anyway.

On that note, it was nice contributing to the Jabber scene,
but I should 
really take my leave. I just thought the OSS/OSI community
was made up 
of people with broader minds, a.k.a Dave. Peter just read
the two emails 
and look at the elegant way in which Dave substantiated his
response, 
instead of "your ideas suck".

There are some things schools can not teach you about
English, I read 
this email about 5 times before I sent it, asking myself,
"how may I 
hurt the other person?" Politicians go to university
for a reason, so 
that they don't screw up international relations by saying
something, 
well, 'unpolitical'. Peter, I really did think more of you.

Note the insertions:

Peter Saint-Andre wrote:
> Jonathan Chayce Dickinson wrote:
>> Hey People,
>>
>> I was perusing over a couple of MSDN articles, when
I came across DIME
>> (Direct Internet Message Encapsulation). Jeez, I
thought, that would fix
>> all the file issues in Jabber (like SI). So I wrote
a draft, any takers?
>> What do you all think? Didn't want to push this
forward until I got some
>> feedback.
> 
> Please don't waste our time with fanciful notions like
this.

Really, ouch!

> 
> I say it is a fanciful notion because:
> 
> 1. DIME was some Microsoft technology.
> 
> 2. Which they submitted to the IETF in 2001 and never
updated.

So what, they abandoned it, it remains viable. The gun was a
huge advent 
in terms of war, it turned knives/swords obsolete, yet you
still find 
them on guns in the form of a bayonet.

> 
> 3. Which they then abandoned in favor of "SOAP
Message Transmission
> Optimization Mechanism" (MTOM).

Why do we have MTOM now? Probably DIME. Note: Optimization.

> 
> 4. Which in turn basically has nothing to do with XMPP
or anything we
> would work on here.

Which, in turn, basically has EVERYTHING to do with
eXtensible (a.k.a 
XML, a.k.a SOAP, a.k.a Messaging, a.k.a *The Internet*)
Messaging and 
Presence Protocol.

> 
> And I don't know why my name is on the protoXEP you
wrote up, because I
> had nothing to do with this.

Once again, sorry, maybe you should remove it from the
template and put 
Romeo there instead as an example.

> 
> Peter
> 

How frustrating. You try to help out: really, I wrote a
whole freaken XEP.

Jonathan Chayce Dickinson

-- 
jonathan chayce dickinson
ruby/c# developer

email:  chayce.zagmail.com
jabber: moitoiinflecto.org

<some profound piece of wisdom>

Oh, one thing I forgot to put to the XEP. The even if it is
small on the 
(compressed) wire, Base64 is large in memory. Keeping it
binary makes 
more sense, it offloads from the XML parser and your app
spends 33% less 
time in loops, which adds up to quite a lot with 10 000
concurrent 
clients. That's the last of it.

Once again Dave, commendable response: I applaud you.

Cheers

(there are two small inserts)

Dave Cridland wrote:
> On Mon Aug 27 20:55:05 2007, Jonathan Chayce Dickinson
wrote:
>> I was perusing over a couple of MSDN articles, when
I came across DIME 
>> (Direct Internet Message Encapsulation). Jeez, I
thought, that would 
>> fix all the file issues in Jabber (like SI). So I
wrote a draft, any 
>> takers? What do you all think? Didn't want to push
this forward until 
>> I got some feedback.
> 
> DIME seems to have been orphaned by its authors, in
favour of 
> soap12-mtom, so I'm not convinced it'd be a good plan
to go this route 
> whatever the merits of the general concept.
> 
> However, as a general note, encapsulating small amounts
of "binary" data 
> in base64 is generally okay - you'll recover the
majority of the 33% 
> bloat through compression (which you hopefully have via
TLS, and may 
> have via other means).
> 
> This isn't suitable for large amounts of data, though,
since compression 
> doesn't fully recover the overhead.
> 
> Moreover, compression algorithms will become
"poisoned" through trying 
> to compress different types of data - typically via
skewing the Huffman 
> encoder and destroying the backreference
"dictionary". This is 
> especially true if the data being transferred is
already compressed, in 
> this case you can observe surprisingly high increases
in data transfer 
> cost (ie, negative compression).
> 
> But sending bulk data through XMPP is a bad idea for a
number of other 
> reasons, not least of which it's trashing the server,
and clogging your 
> XMPP channel (thus reducing its response time).

Look at the interleaving part, it rectifies sending large
amounts of 
data over XMPP (and clogging, or locking, the channel): all
it makes is 
an interesting argument.

> 
> A better method is to negotiate a peer-to-peer session
suited to the 
> kind of transfer you're doing, although I admit that's
not exactly a 
> ground-breaking suggestion. 

But constructive none-the-less. Maybe I will open another
port for 
binary data on my server/farm or make a dime-like protocol.
Thanks for 
the input.

> 
> Dave.

-- 
jonathan chayce dickinson
ruby/c# developer

email:  chayce.zagmail.com
jabber: moitoiinflecto.org

<some profound piece of wisdom>

Please stop top-posting. It makes the discussion hard to
follow.

Jonathan Chayce Dickinson wrote:
> I'm sorry. Didn't mean any harm. 

I know you didn't. I appreciate your enthusiasm, I really
do! But let's
temper it with some realism. 

> I'm not too hot on bandwidth here, 

I'm not too hot on mental bandwidth here, either. 

> so I
> didn't come across MTOM.

So you didn't research very thoroughly, did you?

If you click "I'm Feeling Lucky" at http://www.google.com/
when
searching for "Direct Internet Message
Encapsulation", you come to the
following page:

http://xml.coverp
ages.org/dime.html

And right there at the top you find this text:

Note: Microsoft has listed DIME among "Superseded
Specifications" in its
Messaging Specifications Index Page. See Direct Internet
Message
Encapsulation (DIME) [June 17, 2002]: "This
specification has been
superseded by the SOAP Message Transmission Optimization
Mechanism
(MTOM) specification."

So here we find many things we need to know. In particular,
the
technology was superseded and was never seriously pursued.
So why are we
discussing it here?

However, your suggestion might lead to a productive
discussion about
file transfer / file sharing, see below....

> I wasn't wasting your time, quote: "Didn't want to
push this forward
> until I got some feedback". I don't like wasting
people's time, hell, I
> love helping people out and solving problems: don't
reply to this and
> you will never hear of it again; all it takes is
"nope, not a good
> idea," not something as violent as
"fanciful" (ouch!).

It's fanciful to seriously suggest using a technology that
was
superseded and abandoned over 5 years ago. Let's try to use
more
standardized and widely-available technologies if possible.

> Read the spec and you will see that it has a lot to do
with XMPP: Namely
> (quoted from my draft):
> 
> 1. DIME will rectify the absence of true In-Band
Bytestreams, as it
> defines methods to send multiple payloads in a atomical
message. Each
> payload in the message can reference other payloads in
the message.
> 
> XMPP needs, in my opinion, a way to negotiate real
in-band bytestreams.

What are the problems with XEP-0047?

> Base64 seems like a cop-out to me, it really does.
> 
> 2. Furthermore compression methods used by servers
(which tend to be
> fast instead of -small-effective) will, at best, return
the data to near
> it's original length. If the data is presented to these
algorithms in
> its original state they may be able to compress it even
further.
> 
> Not everyone has a T3 connection at home, some of us
live in South
> Africa, and well, bandwidth here is rather expensive:
and so is a static
> IP (yes, we differentiate between them here, we
/actually/ have
> non-static ones too). When you have to weigh up the
value of 1MB, you
> will know.
> 
> 3. Ultimately this protocol will smooth out the issues
plaguing Jabber
> regarding file sharing.
> 
> This was a hot topic on JDev not too long ago! Demand,
supply. If the
> developers are demanding it, I ASSURE you, the users
will be fast
> following (faster than any cheetah metaphores ;)
> 
> 4. Almost all the existing XEPs should be able to
derive some further
> benefit from this extension (especially Jingle and
SI).
> 
> That is a bit fanciful and ambitious, I admit. Add
XHTML to that list,
> and well there you go, not even nearly all, but 3,
which is a start.
> 
> (Just coming in on that regard, SOAP has nothing to do
with XMPP, but
> neither did SASL (at the start), or XHTML... And there
is SOAP over XMPP
> now)
> 
> And, well, *both* are XML and *both* suffer from the
fact that you can't
> package binary data reliably and efficiently in an XML
document. 

This is very true.

Jabber was not designed for binary transfer. This is a
strength for the
core use cases, but a weakness for things like file transfer
and file
sharing and media sessions, as we have painfully discovered
time and again.

However, personally I'd say we don't want to use an
abandoned technology
to solve the problem.

Another approach would be to define a BEEP transport:

http://www.beepcore.org/


Yes, BEEP is not widely implemented either, but I at least
there are
RFCs (and I am aware of a new push for wider
implementation).

Maybe it would help if I finished writing up some results of
the file
transfer discussions at the recent XMPP DevCon?

> MTOM
> wouldn't exist if it wasn't a real problem. It would
have ended with DIME.

But why was DIME abandoned? (Not that I like MTOM, mind
you!)

> A lot of protocols do it, hell, even your cellphone
does something
> similar (which is why you can call someone and surf
over GPRS/3G/HSDPA
> at the same time).
> 
> In regard to Dave's email, about the negative
compression etc. very
> true, didn't think of that. Nice, constructive
criticism. Dave: the very
> reason I thought of it was because a P2P connection
isn't always
> possible, as in my case (I don't need it, but the
awareness is there). I
> was hoping to use the server to host the
Peer-To-Host-To-Peer connection
> instead of some 3rd party website/ftp server.

That appears to be a sensible line of thinking. However, it
seems to me
that the wide availability of HTTP servers (which could be
bundled with
your favorite XMPP server) makes an HTTP approach even more
appealing.

Part of my question is: do we really need to stream files?
Think about
the difference between media streaming and media
downloading. The use
cases in which true streaming is needed seem few. Even
things like
podcasting are not truly casted in real time -- they are
uploaded to an
HTTP server and then downloaded, to be listened to on
demand.

Similarly, it seems to me that if I want to send you a file,
I could
upload it to an HTTP (associated with my XMPP server), get a
URL for
that file, then send you the URL. If I don't have an HTTP
server at my
disposal, I could fall back to in-band bytestreams. But
let's face it,
the HTTP community has file-upload and file-download down
cold. Why not
leverage all that? We don't need to do *everything* via
XMPP! Do we
really need streaming here, or does it just seem cool?

> I saw a gap, jumped, it seemed like a good idea, but it
obviously
> wasn't. 

Again, nothing is obvious. 

The main point is to solve the file transfer problem once
and for all. I
happy to think that the idea of using DIME to solve the
problem is
far-fatched because DIME is such an obscure technology, but
your
proposal might lead us in the right direction. So thanks for
that. 

> Sorry about putting your name on it Peter: it was in
the
> template, I thought it belonged there. 

Oh gosh no, I'm not that egotistical. 

> I'm sorry I put this out there, I
> could have just as easily made the whole thing a
undocumented protocol
> on the project I am working on, which actually,
ironically, REALLY needs
> it: and probably will land up doing now anyway.

Before diving all the way into DIME, please do reconsider.

> On that note, it was nice contributing to the Jabber
scene, but I should
> really take my leave. I just thought the OSS/OSI
community was made up
> of people with broader minds, a.k.a Dave. Peter just
read the two emails
> and look at the elegant way in which Dave substantiated
his response,
> instead of "your ideas suck".

I never said your ideas suck. I said the notion of using
DIME (an old,
experimental, barely-hatched, superseded, abandoned
technology) was a
bit fanciful. Let's try to use something that we can really
build on.

> There are some things schools can not teach you about
English, I read
> this email about 5 times before I sent it, asking
myself, "how may I
> hurt the other person?" 

Don't take it personally. If you're going to put your ideas
out there,
you need to develop a bit of a thicker skin.

That said, I'm sorry if you felt attacked.

I want to solve the file transfer problem as much as you do
-- this has
been a thorn in our side for 8 years. But we need to do so
in a
sustainable way, with common technologies, using protocols
that have
pluggable libraries that client and server developers can
make use of,
etc. Or so it seems to me.

> Politicians go to university for a reason, so
> that they don't screw up international relations by
saying something,
> well, 'unpolitical'. Peter, I really did think more of
you.

I call a spade a spade. If you think the DIME approach is
worth fighting
for, go ahead and do so. I'm just one person on this list,
and others
may think your proposal has legs. To me, it doesn't seem
like the right
approach.

> Note the insertions:
> 
> Peter Saint-Andre wrote:
>> Jonathan Chayce Dickinson wrote:
>>> Hey People,
>>>
>>> I was perusing over a couple of MSDN articles,
when I came across DIME
>>> (Direct Internet Message Encapsulation). Jeez,
I thought, that would fix
>>> all the file issues in Jabber (like SI). So I
wrote a draft, any takers?
>>> What do you all think? Didn't want to push this
forward until I got some
>>> feedback.
>>
>> Please don't waste our time with fanciful notions
like this.
> 
> Really, ouch!

Hmm, yes, that was overstated. I'm sorry about that. But I
did give some
reasons...

>> I say it is a fanciful notion because:
>>
>> 1. DIME was some Microsoft technology.
>>
>> 2. Which they submitted to the IETF in 2001 and
never updated.
> 
> So what, they abandoned it, it remains viable. The gun
was a huge advent
> in terms of war, it turned knives/swords obsolete, yet
you still find
> them on guns in the form of a bayonet.

An Internet-Draft does not a sustainable technology make.
There have
been thousands upon thousands of Internet-Drafts published.
We need to
base our decisions on something more sustainable than that,
I think.

>> 3. Which they then abandoned in favor of "SOAP
Message Transmission
>> Optimization Mechanism" (MTOM).
> 
> Why do we have MTOM now? Probably DIME. Note:
Optimization.

I don't understand what you mean by that. The W3C probably
produced MTOM
for political reasons. 

>> 4. Which in turn basically has nothing to do with
XMPP or anything we
>> would work on here.
> 
> Which, in turn, basically has EVERYTHING to do with
eXtensible (a.k.a
> XML, a.k.a SOAP, a.k.a Messaging, a.k.a *The Internet*)
Messaging and
> Presence Protocol.

Gosh, SOAP is the world? I think not. 

>> And I don't know why my name is on the protoXEP you
wrote up, because I
>> had nothing to do with this.
> 
> Once again, sorry, maybe you should remove it from the
template and put
> Romeo there instead as an example.

Well, I wrote the template. But I don't write everything
that uses the
template.

> How frustrating. You try to help out: really, I wrote a
whole freaken XEP.

Sorry you're frustrated. Do stick around so we can solve the
file
transfer problem.

Again I think the fundamental question is: do we *really*
need to stream
things, or can we use sender-upload and receiver-download
for many or
most use cases, with in-band bytestreams as the fallback?

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

On 8/28/07, Peter Saint-Andre <stpeterstpeter.im> wrote:
[snip]
>
> That appears to be a sensible line of thinking.
However, it seems to me
> that the wide availability of HTTP servers (which could
be bundled with
> your favorite XMPP server) makes an HTTP approach even
more appealing.
>
> Part of my question is: do we really need to stream
files? Think about
> the difference between media streaming and media
downloading. The use
> cases in which true streaming is needed seem few. Even
things like
> podcasting are not truly casted in real time -- they
are uploaded to an
> HTTP server and then downloaded, to be listened to on
demand.
>
> Similarly, it seems to me that if I want to send you a
file, I could
> upload it to an HTTP (associated with my XMPP server),
get a URL for
> that file, then send you the URL. If I don't have an
HTTP server at my
> disposal, I could fall back to in-band bytestreams. But
let's face it,
> the HTTP community has file-upload and file-download
down cold. Why not
> leverage all that? We don't need to do *everything* via
XMPP! Do we
> really need streaming here, or does it just seem cool?
>
[huge snip]
>
> Again I think the fundamental question is: do we
*really* need to stream
> things, or can we use sender-upload and
receiver-download for many or
> most use cases, with in-band bytestreams as the
fallback?
>
< Peter

Being a only newbie user, with expensive bandwidth, I like
this path.
Currently, I know of disk.jabbim.cz service which provides
HTTP
hosting for files, say <http://disk.jabbim.cz/hdhjabber.cz/>. That is
really convenient for me, either I can tell it to send the
file to
some JID, or I can copy the URL of the file to post
independently.
That's good (except for the Czechy messages, maybe someone
can hack
xml:lang support into it).

The other booster is private storage, sharing the same
hosting space
but the files don't have HTTP access. People can't just
build up the
URLs and download the files from everywhere, abusing the
hosting
service.

I know that this is using SI for sending between JIDs, but
it provides
a nice alternative for people with expensive Internet
access.

Hiếu

> XMPP needs, in my opinion, a way to negotiate real
in-band  
> bytestreams. Base64 seems like a cop-out to me, it
really does.

On the other hand, all XMPP clients already have a BASE64  
implementation -- they have to, in order to deal with SASL
for  
logins.  And even though BASE64 is a bloated method of
encoding, when  
using TLS you already have stream compression which cuts the
size  
back down pretty quickly.

BASE64 may be a cop-out in some ways, but there is logic
behind the  
choice.  Which is not to say that different solutions may
not be  
worth examining, but you end up with a tradeoff between
efficiency  
and ease.  In my experience, if a solution requires everyone
to write  
parsing for Yet Another Standard which has been shoehorned
into XML  
and stuffed into an XMPP stream, you are unlikely to see
terribly  
widespread adoption.

> Not everyone has a T3 connection at home, some of us
live in South  
> Africa, and well, bandwidth here is rather expensive:
and so is a  
> static IP (yes, we differentiate between them here, we
/actually/  
> have non-static ones too). When you have to weigh up
the value of  
> 1MB, you will know.

While low-bandwidth solutions are important, I think file
transfer  
over low-bandwidth connections is not a terribly widespread
usage  
case compared to normal file transfers.  I am not averse to
defining  
some way to do a low-priority, low-bandwidth sending of a
file, but I  
am not sure that should be what general file transfer is
designed  
around.

This is not to say your usage case is invalid.  But neither
is mine,  
and in the case of -fast- versus -size-, I know that for my
own  
situation I am going to want to get that 7 megabyte source
drop from  
my co-worker /quickly/ so I can get back to work, rather
than shaving  
482k off of the total bytes transferred but doubling the
time it  
takes to transfer.

(This is in addition to the negative-compression issues
already  
mentioned by Dave.)

> 3. Ultimately this protocol will smooth out the issues
plaguing  
> Jabber regarding file sharing.

Not setting your sights high there or anything, I see. ;)

More seriously, I do not think there /is/ a silver bullet
for Jabber  
file transfer issues.  There are simply too many usage cases
in  
general, and everyone has a different requirement.  There is
no  
practical way to support every usage case in a single
solution.  (At  
least, not one that I have seen proposed yet.)

> A lot of protocols do it, hell, even your cellphone
does something  
> similar (which is why you can call someone and surf
over GPRS/3G/ 
> HSDPA at the same time).

I digress, but... you cannot surf and call on GPRS at the
same time,  
generally.  This is why if someone calls a GSM phone while
you are  
loading a webpage, they will generally go straight to
voicemail.  (Of  
course, if they call a moment later when the page is done
loading,  
the phone will ring properly.)

I only mention this because as you draw the parallel between
the two,  
I wonder if I am misunderstanding the original suggestion. 
Are you  
suggesting that something similar be done with file
transfers, where  
the message/iq/presence stanzas are placed aside entirely
while the  
stream is used for in-band transfers?

That is an interesting idea for low-bandwidth situations,
though  
seems to me a very high overhead to implement and I am not
certain  
how much practical gain there would be.

> In regard to Dave's email, about the negative
compression etc. very  
> true, didn't think of that. Nice, constructive
criticism. Dave: the  
> very reason I thought of it was because a P2P
connection isn't  
> always possible, as in my case (I don't need it, but
the awareness  
> is there). I was hoping to use the server to host the
Peer-To-Host- 
> To-Peer connection instead of some 3rd party
website/ftp server.

Out of curiosity, if you are planning a server proxy for
peer-to-peer  
links, why is S5B insufficient?  This is not a criticism, I
am  
genuinely curious about why DIME would work better with
server- 
proxying than S5B does.

I would think that even DIME would increase the bandwidth
over S5B in  
a case like that.  After all, if the server is proxying
anyway with  
S5B you can send the data directly, no worries about having
to  
encapsulate the stream of data in anything at all.

You could even define an S5B extension which allows
negotiation of  
stream compression between any of the three points; that
would allow  
legacy clients speaking to a compression-enabled server to
still just  
send the data as-is, and have the server re-compress to send
to the  
bandwidth-limited recipient.  That seems to me to alleviate
bandwidth  
concerns while still keeping backwards compatibility.

-- 
Rachel Blackman <rcbceruleanstudios.com>
Trillian Messenger - http://www.trillianastr
a.com/

> I digress, but... you cannot surf and call on GPRS at
the same time, 
> generally.  This is why if someone calls a GSM phone
while you are 
> loading a webpage, they will generally go straight to
voicemail.
Just to correct you here but they wont goto voicemail (if
you are 
connected to WAP using a dial up method maybe), but you can
definitely 
be downloading via GPRS/3G/HSDPA and still make and receive
calls, i've 
done it myself.


Ive been reading this thread with interest and while I must
agree with 
most of the comments about Jonathan's suggestion expressed
here it has 
stirred up some simple ideas of my own related to file
transfer.

automatic file pre-compression
-----
It would be handy to be able to automatically compress files
before they 
are transferred and then be able to specify this in the SI
negotiation 
allowing the receiving client to then automatically
decompress the file 
once it has received it, this could greatly help with the
size of file 
transfers and would also only need to be done for file types
that would 
benefit from it, i.e. you wouldn't bother with files that
were already 
compressed or things like audio and video files that are
likely to only 
get marginal gains.

jingle bytestream
-----
When we come to implement file transfer using jingle I would
suggest 
that rather than creating a brand new backwards incompatible
file 
transfer protocol that we simply implement a new jingle
bytestream 
transport just like XEP-0047 and XEP-0065 which would allow
complete 
compatibility with the SI negotiation but still gets all the
benefits a 
file transfer over jingle UDP would bring, i.e. better
likelihood of 
connection.

Richard

DNIA 27-08-2007, PN O GODZINIE 21:55 +0200, JONATHAN CHAYCE
DICKINSON
NAPISA?(A):
> I WAS PERUSING OVER A COUPLE OF MSDN ARTICLES, WHEN I
CAME ACROSS
> DIME 
> (DIRECT INTERNET MESSAGE ENCAPSULATION). [...]


WE ALREADY DO HAVE A WAY OF USING A SERVER AS AN
INTERMEDIARY FOR
FILETRANSFERS - IT'S PROXY-65.
IF ONE WANTS TO HOST A FILETRANSFER PROXY AT THE XMPP SERVER
MACHINE,
ONE IS FREE TO DEPLOY PROXY-65.

I REALLY DO NOT LIKE THE IDEA OF FORCING EVERY XMPP SERVER
TO BINARY
DATA TRANSFER INTERMEDIARY.


ON A SIDE NOTE, I AM ACTIVELY FIGHTING THESE USAGE AT MY
SERVER.
AS LONG AS THE SERVER GENERATES LOW TRAFFIC, I AM ABLE TO
RUN IT
CHEAPLY.
IF IT WOULD GENERATE HIGH TRANSFERS, I WOULD BE FORCED TO
PAY FOR OTHER
PEOPLE BANDWIDTH USAGE. NOT NICE.


-- 
TOMASZ STERNA
XIAOKA.COM  HTTP://WWW.XIAOKA.COM/