Freaky FTP Problem

[GreNME]

I'm having an FTP problem here at my office, but I'm having a helluva time figuring out what the source of the problem is.

Computers here are allowed to access FTP (port 21) connecting out to the internet, but there's one address for one of our clients where we continually run into a brick wall. The initial connection to their FTP server works fine, but as soon as a directory listing is sent to view or transfer files, the connection fails every time. This would tell me that the problem is totally on their end, but my personal laptop (which is a MBP) connects just fine, and my own workstation (Windows) connects just fine. This tells me there's a configuration problem on my end, but for the life of me I can't track it down. I've checked the firewalls-- FTP is open. I've checked the routing-- the computers can connect to the remote FTP server, but fail when requesting a directory listing. Passive or active connections-- the remote site prefers passive, but neither work. What throws me through a loop is that these computers connect just fine to other FTP sites for other clients. They just fail on this specific site.

Am I missing something here? It doesn't matter if I'm connecting using Filezilla or the built-in Windows FTP command-line app, I get the same results no matter what.

[aggle-rithm]

I'm having an FTP problem here at my office, but I'm having a helluva time figuring out what the source of the problem is.

Computers here are allowed to access FTP (port 21) connecting out to the internet, but there's one address for one of our clients where we continually run into a brick wall. The initial connection to their FTP server works fine, but as soon as a directory listing is sent to view or transfer files, the connection fails every time. This would tell me that the problem is totally on their end, but my personal laptop (which is a MBP) connects just fine, and my own workstation (Windows) connects just fine. This tells me there's a configuration problem on my end, but for the life of me I can't track it down. I've checked the firewalls-- FTP is open. I've checked the routing-- the computers can connect to the remote FTP server, but fail when requesting a directory listing. Passive or active connections-- the remote site prefers passive, but neither work. What throws me through a loop is that these computers connect just fine to other FTP sites for other clients. They just fail on this specific site.

Am I missing something here? It doesn't matter if I'm connecting using Filezilla or the built-in Windows FTP command-line app, I get the same results no matter what.

You're not using the .Net FTP component, are you? Because it sucks.

ETA: I've found that the .Net component in combination with a web filter causes this sort of problem. That can explain why one application can connect and another can't. Not all FTP implementations are created equally.

[GreNME]

You're not using the .Net FTP component, are you? Because it sucks.

ETA: I've found that the .Net component in combination with a web filter causes this sort of problem. That can explain why one application can connect and another can't. Not all FTP implementations are created equally.

Are you asking about the FTP server? I have no control over the FTP server, but as far as I know it's not using the .Net component.

[Christian Klippel]

FTP uses different ports. Port 21 is only used during the initial connection and as control port, after that almost any other port can be used. Which one is actually more or less random and determined during the handshake. Port 20 is also used as the defualt data port on the server side. See here (http://slacksite.com/other/ftp.html) for some details.

Is the server you are having problems with behind a firewall, or in a DMZ? If so, that might cause the trouble. The wanted ports might be blocked on the other side. Also, there are two modes, active and passive. Active is the standard, passive helps a lot if you are behind firewalls, etc. Have you tried to use passive mode explicitly?

Greetings,

Chris

[GreNME]

FTP uses different ports. Port 21 is only used during the initial connection and as control port, after that almost any other port can be used. Which one is actually more or less random and determined during the handshake. See here (http://slacksite.com/other/ftp.html) for some details.

Yes, I understand that. The connections are failing at the following line:
150 Opening connection for /bin/ls

Is the host you are having problems with behind a firewall, or in a DMZ? If so, that might cause the trouble. The wanted ports might be blocked on the other side. Also, there are two modes, active and passive. Active is the standard, passive helps a lot if you are behind firewalls, etc. Have you tried to use passive mode explicitly?

It's not the firewall, because I turned off the firewall and got the same results. I already know the destination connection prefers passive, and yes I've attempted explicitly passive connections.

[aggle-rithm]

Are you asking about the FTP server? I have no control over the FTP server, but as far as I know it's not using the .Net component.

The .Net component is for the client, sorry.

[Christian Klippel]

Is the ftp server you want to reach actually reachable by other people outside your network? If you want, you can PM me some data and i will try to connect it from here.

And not that i made an edit to my post, which you missed in your reply-quote. For one, you need two ports, 21 and 20, and second, i was speaking of the server being behind a firewall or in a DMZ, not your machine. Sorry for the fuzz.

Greetings,

Chris

[GreNME]

The .Net component is for the client, sorry.

I've attempted with both Filezilla (no idea what they use) and the built-in Windows ftp client (ftp site.address.com) and get the same results. That's all I know.

[GreNME]

Is the ftp server you want to reach actually reachable by other people outside your network? If you want, you can PM me some data and i will try to connect it from here.

And not that i made an edit to my post, which you missed in your reply-quote. For one, you need two ports, 21 and 20, and second, i was speaking of the server being behind a firewall or in a DMZ, not your machine. Sorry for the fuzz.

The site is reachable by people in the network, including those that can't complete a connection. As I went over already in my first post, I've already troubleshot routing, firewalls, and both passive and active connections. Further, I have both a Macintosh and Windows computer in the network that have zero problems connecting. I went line by stinking line in the settings and found nothing different anywhere in the settings.

[kalen]

What flavour of WinDows are you using?

Also, do any of the files in the listing have weird characters in their filenames?

[GreNME]

Windows XP. Filenames are not the problem. It works just fine on two computers on this network (my personal laptop and my computer workstation), but I suspect it has something to do with my computer having full open access to everything on the network. However, my personal laptop has no such open access and is still successfully connecting (but it's a Mac). Nothing about the setup possibilities makes sense, and the umpteen times I've gone over the settings with a fine-toothed comb every setting was exactly the same.

[Christian Klippel]

Are you sure that the machine that has the problems is absolutely clean? There is a possibility that some nasty stuff hijacks the used port and/or connection. Also i suppose that you have no port forwarding done in the router and/or firewall so that the used port's incomming traffic gets sent somewhere else?

Finally, is there a chance that two machines in the network have accidentally the same IP address? I had such a situation by accident once (used DHCP on most machines, but fixed IP on another, so sometimes i got the same IP on two machines). This caused all kinds of strange behaviour.

Other than these, i'm out of options as to what the problem could be.

Greetings,

Chris

[GreNME]

Are you sure that the machine that has the problems is absolutely clean? There is a possibility that some nasty stuff hijacks the used port and/or connection. Also i suppose that you have no port forwarding done in the router and/or firewall so that the used port's incomming traffic gets sent somewhere else?
Finally, is there a chance that two machines in the network have accidentally the same IP address? I had such a situation by accident once (used DHCP on most machines, but fixed IP on another, so sometimes i got the same IP on two machines). This caused all kinds of strange behaviour.

Other than these, i'm out of options as to what the problem could be.

I'm pretty sure that every machine in this building is clean, since that's part of my responsibilities. :) Yeah, no IP conflicts either since the computers in question are all assigned IP addresses per my reservation list. No crossovers

Yeah, I've been out of options for a while now.

Interesting news: I logged into the other computer as myself (I am, after all, Domain Admin), uninstalled the newer version of Filezilla, restarted, installed Filezilla 2.2.25, and after fiddling with the settings a bit I was able to connect and get a directory listing.

I'll definitely be testing this and submitting it as a bug when I figure it out.

[ddt]

Yeah, I've been out of options for a while now.

Interesting news: I logged into the other computer as myself (I am, after all, Domain Admin), uninstalled the newer version of Filezilla, restarted, installed Filezilla 2.2.25, and after fiddling with the settings a bit I was able to connect and get a directory listing.

I'll definitely be testing this and submitting it as a bug when I figure it out.

The other computer meaning one which did or did not have a problem?

I know nothing of the FTP protocol beyond what has already been discussed, but a couple of suggestions nevertheless.

1) Have you used wireshark to see what's actually going over the line?

2) An FTP problem I encountered myself once was at a site that had an IPv6 address, but FTP didn't work properly over IPv6 - about the same way you describe. As I'm running IPv6, you get the picture (and the standard Linux ftp-client does not have an option to force IPv4).

[GreNME]

The other computer meaning one which did or did not have a problem?

The office has several computers. The only one previously able to connect fully was my own workstation (and my personal laptop). All other office computers failed at the directory listing. This currently looks like I found a workaround.

I know nothing of the FTP protocol beyond what has already been discussed, but a couple of suggestions nevertheless.

1) Have you used wireshark to see what's actually going over the line?

2) An FTP problem I encountered myself once was at a site that had an IPv6 address, but FTP didn't work properly over IPv6 - about the same way you describe. As I'm running IPv6, you get the picture (and the standard Linux ftp-client does not have an option to force IPv4).

1) I thought of using Wireshark, and may very well use it in my testing to come, but since I'd have to run it from the same computer I'm trying to connect through, it requires more setup than I've had time to take.

2) I doubt this is an IPv6 problem, but if it is I'll find out when I test it. My guess right now is that there's something different in the negotiation taking place that the newer version of the FTP client is doing that the zFTP server on the other end (I checked the log and the destination server identified as zFTP (http://www.zftpserver.com/)). I hope to find out whether this is the case, and what it is if so.

[deep]

I know you've tested passive mode, but did you test it from the command line as well? In other words, have you typed "passive" and seen the positive acknowledgment?

The reason I ask - it's possible that the FileZilla client is trying to use EPSV (extended passive mode) instead of just PASV, which would cause a problem if the remote server doesn't support EPSV. Or it could be the other way around.

I haven't used the built-in Windows FTP client, but some clients have a "quote" command that will let you enter raw FTP commands (e.g., "quote EPSV", "quote PASV"). If Windows FTP can do that, I would consider trying both of those.

[GreNME]

I don't think the command-line ftp.exe supports a passive mode in XP.

ETA: but you may be correct about the GUI client. I can check that later. I assume the server is looking for PASV as well.

[Fredrik]

The site is reachable by people in the network, including those that can't complete a connection.

I don't understand. Have you actually verified that someone on the internet is able to use the FTP server? If not, you haven't eliminated the possibility that it's a problem with their firewall.

If you want to test passive mode the really nerdy way, you can use a telnet client instead of an FTP client, and type in the commands that an FTP client would send. :)

[~enigma~]

I'm having an FTP problem here at my office, but I'm having a helluva time figuring out what the source of the problem is.

Computers here are allowed to access FTP (port 21) connecting out to the internet, but there's one address for one of our clients where we continually run into a brick wall. The initial connection to their FTP server works fine, but as soon as a directory listing is sent to view or transfer files, the connection fails every time. This would tell me that the problem is totally on their end, but my personal laptop (which is a MBP) connects just fine, and my own workstation (Windows) connects just fine. This tells me there's a configuration problem on my end, but for the life of me I can't track it down. I've checked the firewalls-- FTP is open. I've checked the routing-- the computers can connect to the remote FTP server, but fail when requesting a directory listing. Passive or active connections-- the remote site prefers passive, but neither work. What throws me through a loop is that these computers connect just fine to other FTP sites for other clients. They just fail on this specific site.

Am I missing something here? It doesn't matter if I'm connecting using Filezilla or the built-in Windows FTP command-line app, I get the same results no matter what.
Doesn't FTP mean freaky transfer protocol?

[GreNME]

I don't understand. Have you actually verified that someone on the internet is able to use the FTP server? If not, you haven't eliminated the possibility that it's a problem with their firewall.

Sure I've verified it. Not only did I talk to the guy who's administering the server (who passed along logs of the connections and failures), I even used my own computers at home to test that connections are possible.

If you want to test passive mode the really nerdy way, you can use a telnet client instead of an FTP client, and type in the commands that an FTP client would send. :)

Meh. I'm not concerned with nerd-points. I'd rather figure out what it is that's causing the problem. I'm already pretty sure the passive mode connections work fine on their end, so there's something wrong in my office (and probably with the FTP client we were using).

[Blue Bubble]

I'm having an FTP problem here at my office, but I'm having a helluva time figuring out what the source of the problem is.

Computers here are allowed to access FTP (port 21) connecting out to the internet, but there's one address for one of our clients where we continually run into a brick wall. The initial connection to their FTP server works fine, but as soon as a directory listing is sent to view or transfer files, the connection fails every time. This would tell me that the problem is totally on their end, but my personal laptop (which is a MBP) connects just fine, and my own workstation (Windows) connects just fine. This tells me there's a configuration problem on my end, but for the life of me I can't track it down. I've checked the firewalls-- FTP is open. I've checked the routing-- the computers can connect to the remote FTP server, but fail when requesting a directory listing. Passive or active connections-- the remote site prefers passive, but neither work. What throws me through a loop is that these computers connect just fine to other FTP sites for other clients. They just fail on this specific site.

Am I missing something here? It doesn't matter if I'm connecting using Filezilla or the built-in Windows FTP command-line app, I get the same results no matter what.

What command are you using to view the directory listing ?

"ls" is required to give a list of filenames, one per line, suitable for machine parsing.

"dir" is entirely up to the remote system what form the directory listing is given in. Many (typically Windows) FTP clients wrongly expect "dir" to be the same as "ls", and get totally and utterly confused if the remote system is anything but a Unix (or equivalent) system.

[grmcdorman]

The windows command-line FTP client doesn't do any parsing of the output. It appears that it may be something in the firewall, routing or TCP stacks, possibly, given that multiple clients exhibit the same behaviour.

[GreNME]

What command are you using to view the directory listing ?

"ls" is required to give a list of filenames, one per line, suitable for machine parsing.

"dir" is entirely up to the remote system what form the directory listing is given in. Many (typically Windows) FTP clients wrongly expect "dir" to be the same as "ls", and get totally and utterly confused if the remote system is anything but a Unix (or equivalent) system.

It wasn't the directory listing command that was causing the problem, that was simply the point at where the connection ended. Further investigation showed me that there was a data connection problem that starts after a successful logon and before actual access to data occurs where there was a failure. Something in the way the newer FTP clients handled passive FTP connections was hitting up against a problem (as yet undetermined), and as such the connections were failing until I replaced the version with an older one with more standard (to my understanding of FTP protocols) connections.

-----

The windows command-line FTP client doesn't do any parsing of the output. It appears that it may be something in the firewall, routing or TCP stacks, possibly, given that multiple clients exhibit the same behaviour.

Actually, through checking out the properties of the command-line FTP client for Windows, I found that ftp.exe in Windows XP is unable to handle passive connections and only attempts an active FTP data connection. That right there shows why it failed. Something different in the newer versions of Filezilla was behind the trouble I was having at this FTP site, since reverting to an older version with steps I was more familiar with in the FTP connection process worked fine.

Learning about the lack of passive FTP support in the Windows client was certainly a surprise, though.

[ddt]

Learning about the lack of passive FTP support in the Windows client was certainly a surprise, though.

Thinking about it, it's not so much a surprise for me. The Windows FTP client is, after all, the BSD one. As the command-line has always been of secondary interest in Windows, I can imagine they never updated it with a never version, and so they kept the old BSD-version which didn't yet have support for passive FTP.