Before you start posting possible problems or questions, please know that the following have already been considered:
- I thought Emule already has proxy support?
- I have disabled my proxy settings in Internet Explorer and I want to keep it that way
- My ISP doesn't have a proxy server!
- What if client A goes offline or changes IP?
- Proxy servers can only be reached from inside an ISP!
- Won't ISPs get mad if we abuse their proxy server like this?
- Won't the proxy server overload?
- Proxy servers all have different policies!
Proxy server only cache files smaller than x KB! - Doesn't HTTP give a lot of overhead?
- What extra overhead would this give to the Emule traffic?
- Does this require a major adjustment of Emule?
- What happens to the credit system?
- What about Leechers?
- This only helps popular files!
- This would only cause files to spread quickly within an ISP, not outside it!
My ISP doesn't have a proxy server, or I can't use it for whatever reason. What good is this to me? - Give me something to test the idea with!
- My ISP enforces a maximum download limit (per month). I can't download more anyway.
- There have been several other proposals like multicast, NNTP, Email support. What makes this so different?
- What about compression?
- Encryption?
- But this means Emule won't be Peer2Peer anymore!
- Why HTTP?
- Give me some URLs with some more info!
- I think I see another problem/issue or I have a question.
Q: I thought Emule already has proxy support?
A: That's a totally different sort of proxy support and does not involve caching in any way. It allows eMule to establish a connection to another client or server through a HTTP proxy, but then eMule uses same protocol as without the proxy inbetween. It also does not seem to allow easy transparent proxy support. Our modification does use the HTTP so the transferred data can be cached.
Q: I have disabled my proxy settings in Internet Explorer and I want to keep it that way.
A: Fine. The Emule proxy-setting has nothing to do with the IE setting.
Q: My ISP doesn't have a proxy server!
A: Several ISPs have decided to shutdown their proxies because they stopped being effective since not many customers used them. As webcache-enabled eMules are gaining popularity with every day, the ISPs might re-think the decision. Some ISPs use a "transparent proxy": they already cache everything you download (through http) without you even knowing it. Experimental support for these proxies will be probably included in the 1.1 beta version.
Q: What if client A goes offline or changes IP?
A: So what? As long as the proxy has the file, it can still be downloaded there. It is to be expected that no download attempts will be performed 30 minutes after the proxy source has been created since all clients should have that data after that time.
Q: Proxy servers can only be reached from inside an ISP!
A: Yep... and that's a good thing, otherwise I would still be downloading from a proxy server on the other side of the world. I want to use the one around the corner!
Q: Won't ISPs get mad if we abuse their proxy server like this?
A: Aren't they already mad because we use their up- and downstream like crazy? In fact there are numerous reasons why ISPs will in fact be glad (and the most important one is a big cutdown on traffic costs)!
Q: Won't the proxy server overload?
A: We only transfer data through a webcache if we are quite sure that somebody else will want to have it. We can instruct the proxy servers to only cache the files for a limited amount of time (tens of minutes). And we'll try to avoid having the same file cached under different names on the same proxy server. Also if the webcache is known not to be caching inbound (ISP-internal) traffic, it will be used for outbound traffic only. Also, there is an identifier in the URL which allows blocking the eMule traffic on the webcache if the owner wants to do so.
Q: Proxy servers all have different policies!
Q: Proxy server only cache files smaller than x KB!
A: Yep, and some might not work or may need some tweaking to get them to cooperate. But with a few simple tests we could at least make sure it works for all major ISPs.
Q: Doesn't HTTP give a lot of overhead?
A: HTTP uses a few plain text headers, while Emule uses short binary opcodes. But we'll still be using every aspect of Emule and only Emule decides when the time is right to use a HTTP connection. And once a file is cached and downloaded twice, the small extra overhead has been payed back.
Q: What extra overhead would this give to the Emule traffic?
A: Not very much. All clients need to know the proxy server of all other clients (about 35 extra bytes of overhead every time an unknown webcache-enabled client starts communicating with your emule). Any other overhead only starts as soon as a new-proxy-source is generated, but that overhead is paid back _very_ quickly once it becomes a positive-proxy-source. Low investment in overhead; very big and immediate payback.
Q: Does this require a major adjustment of Emule?
A: Not really. Every thing that works now keeps working and remains backwards-compatible. It's just an add-on. An alternative way to do the actual downloading of a filepart.
Q: What happens to the credit system?
A: For the first time, it remains in place, but no extra credits will be granted for webcached uploads to prevent exploits. Also credit system might be deactivated once a significant amount of users switches to webcache-enabled clients. The credit system has negative effect on the network and should not be active when they take over the hand due to the fact that the good effects of the credit system will be outweighed by the boost brought by the webcaches.
Q: What about Leechers?
A: The "problem" of leechers won't go away (but won't increase either). The "negative effect" that leechers have on the network _will_ be reduced! A leecher that downloads a cached file does not negatively affect the network.
Q: This only helps popular files!
A: In a direct sense: yes. But those also form the biggest load on the network, so by taking a big chunk of that load away, all other traffic gets more room: more bandwidth and shorter waiting times in the queue. The chance that 2 clients are behind the same proxy server is higher for files that are of regional interest only (that are in German e.g.). So if that's the reason why they are "rare", there's still a good chance that the caching would work.
Q: This would only cause files to spread quickly within an ISP, not outside it!
Q: My ISP doesn't have a proxy server, or I can't use it for whatever reason. What good is this to me?
A: Currently (without cache): there is 1 user in ISP X that has the file, so you have 1 source to download it from. With caching enabled, 100 users quickly have it (with no disadvantages to you) so you get 100 sources to download it from.
Q: Give me something to test the idea with!
A: You can simply test your proxy server using telnet. Telnet to your proxy server, instruct it what file you want and it will give you a cached copy or otherwise will try to get it for you.
An even easier way is using the program 'wget'. In a command-prompt: set the http_proxy environment variable to your proxy server (and :portnumber) and then type wget -s http://80.56.135.209:82/ to download a testfile.
Q: My ISP enforces a maximum download limit (per month). I can't download more anyway.
A: Depending on how your ISP does it, you might be in luck: some (most?) ISPs do _not_ count traffic from their proxy server. This is their way of convincing you to start using the proxy server in the first place. If your ISP does count this traffic, you might want to contact them and point out to them that this is not fair, nor wise on their part.
Q: There have been several other proposals like multicast, NNTP, Email support. What makes this so different?
A: Multicast would be nice too, but most ISPs don't support it. Using NNTP (News) and Email to send files is hopelessly inefficient and would give a lot of overhead and problems. Lot's of stuff that can go wrong and eventually it could piss off the ISP because you're abusing a service not intended for file transport. On the other hand, there are email companies like GMX who offer you a 1GB inbox and even have extra online file sharing functionality for free.
Q: What about compression?
A: Whether files are sent through a proxy are compressed or not doesn't matter at all. In fact chances are you just downloaded this forum gzipped. HTTP supports gzipped content, but we don't even need to use that.
Q: Encryption?
A: There is a high risk that users who spread copyrighted contents will use our feature, which can lead to legal attacks on ISPs, which might force them to shutdown their webcaches. Since we don't want them to, encryption is implemented (in version 1.1 beta). Now that the ISPs cannot say whether data on the webcache is copyrighted or not, they cannot be forced to disable the proxies.
Q: But this means Emule won't be Peer2Peer anymore!!
A: P2P is only a virtual 'thing' anyway. Real Peer2Peer would only be possible in LAN situations or with null-modem connections. A P2P network like the Emule-network sits on top of a WAN (internet), just like the WWW we all surf uses the internet. I don't see the difference between downloading something from a user directly or downloading it (much faster) from a cache that works on behalf of that same user. Remember that caching is used everywhere inside and outside your computer. In fact the data you download using Emule is already cached: by the Windows (or Linux) disc-cache and the memory caches in your CPU and memory banks.
Q: Why HTTP?
A: The guys that thought up the HTTP protocol used by the WWW were pretty clever and thought of several ways to facilitate caching. It's an already well-esteblished standard; no need to reinvent the wheel. Proxy servers are already in place and running, would you be willing to pay for dedicated Emule-cache-servers?
Q: Give me some URLs with some more info!
A: http://www.mnot.net/cache_docs/
Q: I think I see another problem/issue or I have a question.
A: Feel free to post it in the forum or pm/mail a member.