Title says most of it. I use ES File Explorer, but the issue seems to exist in any file manager. Copying files to the SFTP server nets 20-30MB/s, whereas copying files from is ~400-500kB/s.
I gather this may have something to do with Android treating these copy operations as a background download, but that's mostly supposition.
p.s I had similar problem when both my laptop and my phone where connected via wireless, and the download speed on my phone was very slow (phone downloader/laptop uploader over wifi).
wifi is a shared volume, and only one client can talk at the time.
– Rabin May 19 '14 at 09:20