Europa File Transfer

Cyberteams TRECIS
1

Hello,

I am unable to connect to Europa using SFTP, FTP, FTPS, or SCP. I’ve tried using FileZilla, Mobaxterm, and connecting directly from a terminal.

This seems to be a new problem, because previously I’ve been able to send files to Europa from Ganymede using SFTP. But now if I try that the connections fails and I get “Received message too long 976887913” instead of an actual error.

I can send a verbose debug log from FileZilla if it will help.
Thanks,

2

Are you accessing europa.circ.utdallas.edu and are you on the VPN?

I can log in as you on Europa.

Best,
Chris

3

I can SSH in just fine (when using the VPN). The issue seems to be solely with file transfer protocols. Mobaxterm has an embedded file transfer system, but even that one doesn’t work when the adjoining SSH is connected fine.

Thanks,
Patrick

4

I think I figured out the problem; can you try one more time?

5

I still can’t get through. I tried FileZilla, Mobaxterm, and terminal SFTP again. All with the same errors.

6

Ok one more thing to try, and if this doesn’t work, I’ll fire up a windows VM tomorrow and install that software you referenced and try it.

First can you confirm you’re able to sftp from the CLI from ganymede.utdallas.edu to europa.circ.utdallas.edu? It should not have worked earlier but should work now. If that works but your GUI clients don’t, I have one more idea.

There is a long standing issue for some SFTP clients and openssh; when ~/.bashrc is sourced, if it prints something to STDOUT or STDERR, it will cause the SFTP to quit with that “message too long” error.

Trying adding this to the top of your ~/.bashrc file right below #/bin/bash on line 2 and before the “source global …” entry.

[[ $- == *i* ]] || return

or temporarily commenting out things that print to STDOUT like “ml restore” and/or sourcing the intel compiler scripts.

Let me know what happens when you try both and if they don’t work, I’ll go play with those GUI clients directly.

7

Using SFTP via the Ganymede CLI still didn’t work.

But you were right about the .bashrc file. The only change I’d made since the last time I’d done any file transfers was adding a line to source the Intel API. I commented out that line, and now file transfers work again using any method.

Thanks for the help,
Patrick

8

Oh I know what’s causing the europa from ganymede problem.

By default, openhpc put the following entry in ~/.ssh/config

# Added by Warewulf  YYYY-MM-DD
Host *
   IdentityFile ~/.ssh/cluster
   StrictHostKeyChecking=no

This is causing ganymede to force it to use the ~/.ssh/cluster.pub public key to log into europa. However, your SSH keys are different on the two machines.

We haven’t had to support multiple systems that use different $HOME filesystems so we’re still working out the kinks.

I’ve manually put in a fix for you using ssh keys and you can now sftp from ganymede to europa.

[prc160030@ganymede ~]$ pwd
/home/prc160030
[prc160030@ganymede ~]$ whoami
prc160030
[prc160030@ganymede ~]$ sftp europa.circ.utdallas.edu
 _________________________________________________________________________
/\                                                                        \
\_| Use of UTD Information Systems is subject to the UTD Information      |
  | Security and Acceptable Use Policy.  Pursuant to Texas Administrative |
  | Code 202:                                                             |
  |                                                                       |
  | (1) Unauthorized use is prohibited;                                   |
  | (2) Usage may be subject to security testing and monitoring;          |
  | (3) Misuse is subject to criminal prosecution; and                    |
  | (4) Users have no expectation of privacy except as otherwise provided |
  |     by applicable privacy laws.                                       |
  |   ____________________________________________________________________|_
   \_/______________________________________________________________________/
Connected to europa.circ.utdallas.edu.
sftp> ls
bin         potentials  scratch     templates   
sftp>

Tomorrow, I’ll write up a little primer on what I did and post it as a thread and I’ll submit a bug report so that this gets fixed in the next feature update to europa.

Thanks for all the feedback today!

Best,
Chris

9

Hi all,

I have modified the ~/.bashrc file as follows, and file transfer by using scp works well.

#source /opt/intel/oneapi/setvars.sh
if [ ! -z “$PS1” ]; then
source /opt/intel/oneapi/setvars.sh
fi