The Jabber Project: File Transfers

The Jabber Project:
File Transfers (DRAFT!)
by Jeremie <jeremie@jabber.org>

Introduction

One of the key and critical features needed by Jabber is file transfers. The Jabber protocol and architecture isn't designed to handle this kind of bandwidth or usage internally, so this proposal attempts to provide a mechanism to have standard file transfers become part of Jabber and maintain interoperability with other IM systems' file transfer functionality.

This proposal relies on the info/query proposal.

HTTP

The most common and standard way of transfering files existing on the Internet today is unmistakeably HTTP. Why not utilize all that has been invested in knowledge and development in HTTP for file transfers with Jabber?

What are the drawbacks:

Clients supporting file transfers must open another outgoing socket
A rudimentary understanding of HTTP must be implimented in supporting clients
Additional tools might need to be run on the servers and transports

What are the advantages:

MIME and other file meta-information is handled within the HTTP headers
HTTP supports byte-level download recovery for broken downloads
Communications are plain-text over simple tcp/ip sockets
URL's are standard and known ways of expressing HTTP accessable resources
Plethora of existing code for handling HTTP

How?

It's not as simple as just saying "Ok, we'll use HTTP". How?

First, the HTTP resource must be expressed in a URL, and this URL must be communicated from one client to another. One possibility for this would be to add a type='file' to the message packet, and restrict the say tags to containing the URL of the file. This would have the advantage that if a user is using a client that doesn't support type='file', they would see the message and could cut/paste the URL into a browser. But what if the sending user wanted to also include a description of the download? Two possibilities, the say field could include the description AND URL and the client software would have to scan and parse out the URL, or as another option, the URL could be placed somewhere else in the message(possibly in the ext tags?). This needs some discussion.

Second, how does the client know anything about the file before it offers the user a download/save prompt? Simple, use the standard HTTP HEAD method to retreive the headers and meta-information about the file.

Sending Files

So how does the sending client send a file? This is the tricky part where things can start go get complicated.

Client to Client! The most efficient way to get files from one client to another is directly from one to another. This of course has various drawbacks: the sending client must impliment a small HTTP server, must open a listening socket on the users system, and must expose the users IP address to other users. Lightweight HTTP servers are easy to impliment, and if a user is sending a file to another friend they probably don't mind the temporary security exposure to them. The other possibility is that the receiving user could open a listening socket, and the sending client could perform an HTTP PUT of the file on to the recipients computer. This option would have to be negotiated between the two clients and presented as an option to both users, the negotiation would happen via the info packet described in the info/query proposal.

How do you do it without client to client? Simplest option: The server can run a small background daemon called socketstreamer. This daemon has no other purpose in life other than to connect one socket to another and copy data back and forth between them. Again, using the info packet, the client would need to negotiate with the server to see if there is a socketstreamer available on the server and which port to use.

Extended functionality: Using the HTTP approach allows a nice new feature that could be very handy. Another special daemon is running on the server(or, more likely, it's a special module for apache). This daemon is configured to allow users to upload files to it via HTTP PUT to their own area(with a configurable disk quota of course). A user could then put the file on the server and announce it's location to friends after it's uploaded. The advantages to this are that other users could download the file after the sender is offline or download it multiple times from multiple locations. Also, the file(s) can be group sent so that 15 friends can download it from one user that only has a modem connection(again, quotas on bandwidth would also be an option). The user could also upload files that they send to other locations, such as from home to work. If file transfers work our according to this proposal, I'd expect this to be a popular bonus feature for servers to offer.

Transports and other IM Systems

Great, but for example, how does a file get from an ICQ client to a Jabber client?

Since the ICQ transport is acting as an ICQ client, it would need to either have a special daemon running or fork one for the file request. After the IM system file transfer negotiation has happened, the transports background daemon would listen on a socket for an HTTP GET from the Jabber client and operate similiar to the socketstreamer above to copy bytes from one socket to another.

Any specific questions about this document or Jabber can be directed to the Jabber Development Team at <team@jabber.org>.